Increased Risk

It should have been a road trip, travelling the U.S. with a rental car for two weeks. But when Sven Drewert*, in his late 30s, wants to increase his credit card limit for the holidays, he experiences a surprise. The bank rejects this increase. He is told that his Schufa score is too bad, even though he has always paid his bills and in any other respect leads a rather inconspicuous economic life.

^* Name changed

What kind of institution is this to pass a judgement on almost every consumer with its scores? It is not a public authority, as some may think. Schufa Holding AG is a private enterprise, a so-called credit reference agency. Banks, mobile companies and online retailers who rely on trusted information about potential customers for their business base their assessments on this institution’s evaluation. Schufa is not a monopolist: also other credit bureaus such as Crif Bürgel, Creditreform Boniversum or Arvato Infoscore collect data from individuals and companies and evaluate their creditworthiness. But no other credit agency is as well known as Schufa, which describes itself as the market leader.

Sven Drewert learned about Schufa’s influence already as a student. The local Internet provider refused to conclude a DSL contract with him, also referring to his Schufa score. Indeed, without Schufa information any participation in social life is hardly imaginable: no mobile phone contract, no rented accommodation, no bank loan, no paying on account in the online store. The company claims to have information on more than 67 million consumers. This information includes negative data such as payment defaults or entries from the list of debtors. But also so-called „positive information“, about credit cards, current accounts or real-estate loans. Many of these data are transmitted by banks and companies.

Schufa has stored only positive features of Sven Drewert, and yet the evaluation transferred to the bank by Schufa read „satisfactory to risk“. Credit institutions do not only base their decisions on possible business partners on the Schufa score, but all too often it may lead to problems. Mr Drewert was lucky, his girlfriend was able to help out with her credit card for the trip to the United States. But he still does not know to date why Schufa evaluates his risk to be at such a high level. How exactly Schufa arrives at its evaluation is a business secret. You could also call it non-transparent. The algorithm is similar to a black box: It is only vaguely possible to understand how it works - until now.

Journalists of Bayerischer Rundfunk and Spiegel were able to evaluate the Schufa data of more than 2,000 consumers. We will not reveal a secret Schufa formula. This dataset represents only a small, distorted section of German consumers: men are overrepresented, older people are underrepresented. Nevertheless, the data provide an unprecedented insight into the heart of Germany's most famous credit agency and allow us to draw important conclusions with regard to the functioning of Schufa scoring. They show that Schufa knows far less about many people than one might think - and yet dares to calculate an exact score. The data also indicate how age and gender could affect the risk assessment. And what disadvantages consumers might have if companies use an old score version.

Positive features, negative rating

Schufa itself states: They have only stored positive contract information of more than 90 percent of all consumers. „And that's a good thing“, says the Schufa website, because that could make life easier for the consumer. For people like Sven Drewert that must sound like mockery. That is because positive features are still far from a perfect score. Very much the opposite is true: Of those consumers in the dataset to whom only positive features apply, Schufa certifies every eighth of these consumers to have a „satisfactory to increased“ or even high risk in the Schufa base score. Schufa will not let the customers know whether this evaluation is based on the number of their current accounts, or maybe even the second credit card tipped the scales.

It is also striking that Schufa knows surprisingly little about many people. For almost a quarter of the persons in the dataset, Schufa has stored a maximum of three pieces of business information – for example, the opening of a current account and the conclusion of a credit card and a mobile phone contract. Otherwise it only has general data such as addresses, age and gender. And although Schufa knows so little in some cases, it sets ratings that may be a problem for consumers.

A 20-year-old employee who provided his credit report to the project „OpenSchufa“ learned what this can mean. He is certified by Schufa to have an „increased to high risk“. In addition to his age, gender and the number of his addresses, its evaluation relies solely on the existence of three current accounts, one of which has existed for several years.

This applies not only to him: The data set contains more than 20 consumers whom Schufa certifies to have a „satisfactory to increased risk“, even though their financial history does not include more than three entries - only positive ones. Schufa does not know much more about Sven Drewert, who was denied a higher credit card limit because of this assessment: In his credit report, three current accounts are recorded, one of them with credit card. That's apparently sufficient for Schufa to declare him a risk case.

Small difference, big effect

It's actually wrong to call it Schufa score. Schufa calculates different values for different industries such as banks, telecommunications companies or mail-order companies. And over the years, new versions of these industry scores have been added, in which Schufa has adapted its formulas. At the beginning of 2018, three different versions were used. Moreover, companies can have their individual Schufa score, adjusted to their needs. Basically, all scores work similarly.

From the combination of their respective stored data, Schufa calculates a value between 0 and 10,000 points for one person. How exactly which feature is included in the calculation is secret, the weighting differs from industry to industry and from version to version. One thing is clear: the higher the value, the better. In an information from the record, Schufa calculates a score of about 9,493 points for the banking industry. That does not sound so bad, but Schufa concludes from this that his repayment probability, i.e. the probability with which he makes payments in accordance with the contract, is 93.79 %. In words this means a „satisfactory to increased risk“.

Had the man only achieved two more points, this would mean a percentage of 95.62 % and he would only have a „low to manageable risk“. But what many do not know is that for each score variant only a limited number of about 15 different repayment probabilities is transmitted. In this case the figure is about 93.79 % or 95.62 %. Ratings in between, e.g. a probability of 94 %, do not exist.

The consumer usually does not hear anything about that. Schufa has hard limits and does not state any statistical error margins. Small details can lead to a consumer slipping into the next worse category, thus getting a more expensive loan or having to opt for a prepaid mobile phone, because the provider does not want to give him a contract.

Young, male, above-average risk

But not only information about current accounts and mobile phone contracts can play a role. Particularly in older score versions, the Schufa rating is also affected by features which cannot be influenced: A 25-year-old man has to expect a different rating than a 65-year-old woman.

This can be read on the so-called „risk in the data types“, which is shown in the Schufa information. Right here, under „General Data“, Schufa assesses consumers' risks based on information such as date of birth, gender and number of stored addresses. Whoever has a minus sign in his/her information here is certified an „above-average risk“ in this category by Schufa. In the dataset, for example, younger people are often rated worse than older people. Although they have otherwise similar features. It is also apparent that in this category minus signs are more frequently found in males than in females. With young men, this occurs very frequently in the dataset.

Younger men more often have bad ratings than older ones

Classifications in the category „General Data“ across different age groups. Population: all male consumers in the dataset with less than three relocations.

The data also suggest that many relocations have a negative impact on the rating. Nevertheless, even with the more than 2,000 available ratings, it cannot be concluded how exactly the factors affect the respective score and whether Schufa includes them individually in the calculation or combines them.

The fact that age and gender are included in the score is actually not prohibited, as long as there are statistically measurable differences. Schufa emphasises to use only such information permitted by law for the calculation of credit scores. The General Equal Treatment Act which aims to protect consumers from discrimination based on age and gender is not effective with regard to credit bureaus.

In the jungle of versions

After all it is true that in certain industry scores, the information on age and gender are no longer included. The latest generation of Schufa scores, „Version 3.0“, was introduced in 2017 and for some industries only takes into account information from a consumer's business life. The new version offers more accurate predictions, Schufa writes on its website.

But the data evaluation shows that many companies apparently do not use the allegedly more accurate forecasts yet. In the present data, companies requested a majority of old score versions in 2018. They can still retrieve scores of version 2 whose formulas have been in use for ten years now. Up to June of this year also version 1 was used, which had been introduced by Schufa in 2001.

This is not a disadvantage for consumers per se. The data evaluation shows that some consumers have a better rating in the old score versions than in the new version 3. It is also possible to have results exactly the other way around: A car-sharing provider, for example, always asked for an old score of the trade sector for its customers. In the most extreme case, the company got the information that the consumer is a „very high risk“, although the latest score version sees only a „small to manageable risk“.

There are these differences even with the bank score. Which score version a bank actually uses is not obvious to consumers before completing a deal. The extent to which the calculation methods differ is shown by juxtaposing the different versions of the bank scores of all persons in the dataset: In the following chart, each line links the repayment probabilities of the same person – on the left side the bank score according to version 2, on the right side in the current version 3:

Bank scores vary to such a major extent between versions

A line connects the repayment probabilities of one person in the various versions of the bank score: Very low risk, low to manageable risk, satisfactory to increased risk, significantly increased to high risk.

Schufa and its customers point out that the submitted score is just one of many decision criteria when it comes to granting a loan. Companies can also use the positive and negative features stored by Schufa directly. And often a bank has known the person for some time and knows how much salary is received in the respective account each month.

Bank insiders who want to remain anonymous, however confirm that only a too low Schufa score can cause the bank to reject a loan. Also in the case of Sven Drewert, who intended to travel to the U.S., the bank referred to his Schufa score when denying him the higher credit card limit.

We confronted Schufa with the findings of our evaluation and asked them to comment on the matter. We asked about the different score versions and the effects of age and gender on the score. We wanted to know why even people without negative features get bad ratings and which uncertainty scores are associated with, which rely only on little information. Although Schufa responded in a nine-page letter, they do not want us to quote from it or reproduce the contents correspondingly.

Is a better scoring possible?

Consumer and data protection specialists have been calling for more transparency on the part of Schufa and other credit bureaus for years. „Scoring providers shall disclose to consumers the essential features on which they are scored and, if possible, their weighting in a comprehensible and plausible manner“, says the recently published report of the German Council of Experts on Consumer Affairs, an advisory body of the Federal Ministry of Justice and for consumer protection. However, business secrets should not be violated by this.

Professor Gerd Gigerenzer is co-author of the report. Credit bureaus fulfil an important function, says the renowned scientist. But he also says, „The time has come for us to put the interests of consumers above the business interests of the credit bureaus.“ The report issued by the Council of Experts also calls for a better control of the scoring procedures by the data protection authorities, including research by Bayerischer Rundfunk from spring.

In May 2018, BR reporters had shown that the review of scoring procedures by data protection authorities was based to a large extent on reports commissioned by the credit bureaus themselves from universities and researchers. As a solution, the German Council of Economic Experts proposes a state-owned digital agency that independently checks how good algorithms really are.

The report also states: „Above all, however, scoring must be understandable for the concerned persons“. Professor Gigerenzer puts it this way: „Personally, I think we need a change here. The change must be in that to give consumers the right to understand how their value comes about.“