Case study · Data/Decision Analytics · Public Kaggle credit data, treated as a lender's loan history

Credit Risk Analysis - Where is a lender losing money it didn't have to?

A lender hands over its loan history and asks one thing: how do we stop losing money on loans that could have been avoided? I investigated the data to identify where default concentrates, tested whether simple lending rules were sufficient, and evaluated whether a predictive model could improve the decision. I then translated the findings into operational recommendations.

Code on GitHub

Part 1. The evidence and the leads

The data, and what I did to it

The dataset holds 32,581 loans with 12 attributes covering borrower profile, loan details and repayment outcome. I checked it as I would a client's data before drawing any conclusion. After cleaning, 32,411 loans remained.

The underlying public dataset does not specify a currency. For presentation purposes, all monetary values in this case study are expressed in R as an illustrative convention and should not be interpreted as actual lender losses or South African rand observations from the source data.

Lead 1. The size of the problem

21.87% of loans defaulted. They account for R76.97M of the R310.94M lent (24.8%). Defaulted loans were also larger on average, with an average loan amount of R10,857, compared with R9,240 for loans that did not default.

Lead 2. An affordability cliff

Default sits between 12% and 22% while a loan is under 30% of income, then jumps to about 70%. The cliff appears in every income band: for the lowest earners default goes from 30% to 82%, and for the next band from 21% to 69%. About 3,800 loans (12% of the book) sit above the line and produce roughly 38% of all defaults.

So what: a loan-to-income check at application is cheap and easy to explain. It should trigger a review, not an automatic decline, since about 30% of those borrowers still repay.

Lead 3. Income, with a floor effect

Default falls steadily as income rises: 45.6% under R30k, 28.0%, 19.6%, 15.7%, then 9.4% above R80k, where it stops improving. Borrowers under R45k are 36% of loans and about 57% of defaults. Income and affordability each carry their own signal and they compound: the same book contains loans that default at 7.8% (high income, low ratio) and at 82.4% (low income, high ratio).

So what: extra scrutiny belongs at the low end of the income scale. Above about R80k, more income buys little extra safety.

Lead 4. The lender's own grade works

Default climbs from 10% at grade A to 59% at D and 98% at G. Grades D to G are 15% of loans and about 42% of defaults.

So what: the existing grading does real work, so the question is what it misses. Grade is set by the lender partly on perceived risk, so it confirms the lender's judgement more than it reveals new risk.

Lead 5. Housing and purpose

Renters default at 31.6%, against 12.6% for mortgage holders and 7.5% for owners. By purpose, debt consolidation (28.7%), medical (26.8%) and home improvement (26.2%) default far more than venture (14.9%) or education (17.3%).

So what: people consolidating debt are often already stretched, so affordability checks matter most there. Housing status is useful context for reviewers, but it needs a fairness check before it goes into any rule.

Two leads that didn't hold up

Default stayed between 20% and 23% in every age band, and across credit history lengths until the longest band, which holds only 101 loans. Neither separates good borrowers from bad.

So what: the lender gains nothing from using age, which also avoids a compliance problem.

What Part 1 told me

Every lead is real but partial, and the strong ones overlap. A lender could stack rules, but someone has to decide the weights and how they combine. A simple two-rule filter (loan above 30% of income, or grade D to G) catches 71% of defaulters while flagging a quarter of applicants. The remaining question was whether a model could find more of the rest without flagging many more good customers.

Part 2. Can a model beat the rules?

A supervised model learns the weighting from past borrowers who did and did not default, so nobody has to set it by hand. I trained a LightGBM classifier on 80% of the loans, with class weights because defaulters are only 22% of the book, and tested it against the rule filter on the held-out 20%: 6,483 loans, 1,418 of them defaults.

The review workload is about the same, yet the model finds 190 more defaulters and wrongly flags 78 fewer good customers. The rules are not weak: they catch 71% with no model at all. The model adds the last 13 points, the borrowers who pass each simple test but are risky in combination. AUC was 0.946, and accuracy at this threshold was 88.6%.

What the model relies on

Grouped by importance (gain): affordability and income about 43%, the lender's own pricing (interest rate and grade) about 31%, housing 11%, purpose 9%. Age (1.9%) and credit history length (0.4%) barely register, which confirms the two dead ends.

So what: default here is driven mainly by the borrower's capacity to repay. Neither age nor history length is worth building a rule around.

The decision: where to draw the line

For the model comparison, I used a 0.40 threshold to compare performance at a review workload similar to the simple rule filter. I then treated the threshold as a business decision and tested alternative thresholds against assumed costs. To put numbers on it, I assumed a missed defaulter costs 60% of an average defaulted loan (about R6,514) and a wrongly flagged good customer costs one year of interest on an average loan (about R1,016). On those assumptions, screening nothing costs about R9.24M, the rule filter R3.27M and the model R1.95M: 79% below screening nothing and about 40% below the rules.

The cost-minimising threshold depends on those assumptions. Across nine scenarios it ranged from 0.25 to 0.50:

The more a lender loses per default, the lower the threshold should be. The more it earns on each good loan, the higher. So the threshold is a business setting, not a model property, and it also sets the review workload: at 0.25 about 46% of applicants are flagged, at 0.50 about 23%.

Stress test: what if the lender's pricing isn't available?

Interest rate and grade are set after the lender has assessed the borrower, so a model scoring a fresh application wouldn't have them. I removed both and retrained.

Recall barely moved. The cost was more false alarms: 205 more good customers flagged. The model can be constructed using information available at application time, with a heavier review queue, and it catches more than twice as many defaulters as a rule a lender could apply on day one.

Recommended operational changes

Under the central 60% loss assumption and 11% interest-forgone assumption, 0.35 is the cost-minimising threshold. I use that threshold for the illustrative operating tiers below.

Share of applicants in each tier, with the default rate inside it (6,483 test loans).

1. Tier the application process. Two-thirds of applicants move to fast approval at a 4.4% default rate. Analysts review the 16% in the middle. The 19% scored as high risk, of whom 85% default, get collateral, a guarantor, or a decline.
2. Make affordability the first gate. Default jumps from about 20% to about 70% past 30% of income. On its own the check catches only 38% of defaulters, so it works as a trigger for review, not a complete policy.
3. Review lending to grades D to G. They are 15% of loans and about 42% of defaults.
4. Use the model as decision support. A person makes the final call.
5. Retrain on the lender's own data, set the threshold from its own recovery rate and margins, and monitor drift.

Limits of this case

• The data is a public simulation, and a lender's own patterns may differ.
• Threshold selection and final tier construction should be repeated on a validation set or through cross-validation, followed by a genuinely untouched test set.
• Cost figures are my assumptions. The data has no recovery or currency information.
• The operating tiers were derived from the application-time model and evaluated on the held-out test set.
• Age and housing status may provide useful context for reviewers, but both require a fairness and compliance review before being used in any lending rule.

If this were a real engagement

I would validate on the lender's own history, agree the cost assumptions with its credit team, and integrate the tiers into the application workflow.

Power BI dashboard for the lender's credit team: default rate by affordability, income, purpose and housing.

Disclaimer

This case study demonstrates how I use data, statistical analysis and machine learning to investigate business problems and support operational decisions. Rather than treating the model as the end product, I use it as part of an investigation: identify where risk concentrates, test the assumptions behind potential interventions, evaluate whether a more sophisticated approach adds value, and translate the evidence into practical changes to the decision process.