You applied. The screen thought for four hundred milliseconds. It said no. And when you asked why, it had nothing to say — because nothing inside it speaks your language. A model with tens of thousands of parameters just decided you don't get the loan, the card, the apartment, the rate — and there is no sentence anywhere in it that reads "denied because X." There is only a number crossing a threshold. You were declined by a machine that cannot, even in principle, tell you why.
That is not a bug. That is the design. And you should understand exactly what was done to you, because the same architecture is quietly moving from lending into everything.
The old score was legible. The new one is a fog
The classic credit score was crude but honest. FICO, the score most of the West runs on, is built on roughly five ingredients you could name: payment history, amounts owed, length of history, new credit, credit mix. Weighted, published in rough proportion — payment history around 35 percent, utilization around 30. You could look at your file, see the late payment from three years ago, and know what to fix. The logic was legible. You could argue with it.
The new generation does not work like that. Lenders and fintechs now feed machine-learning models — gradient-boosted trees, neural nets — on thousands of "alternative data" features. Not just your payments. Your phone model. The time of day you apply. How fast you scroll the terms. Whether your email is gmail or a custom domain. Your device's battery level, seen by some lenders as a proxy for financial stability. The gaps between your keystrokes. Each weak signal, alone, means nothing. Braided by the model into a high-dimensional surface, they produce a verdict — and no human, including the engineer who trained it, can point to the reason. The model does not have reasons. It has weights.
The black box is not an accident
Here is the part the industry does not advertise. The opacity is convenient.
A legible rule can be audited. If the rule says "we charge more for zip code 60624," a regulator sees redlining and fines you. But if a neural net charges more for people in that zip code — having learned it from ten thousand correlated features nobody explicitly encoded — the lender shrugs: the model decided, we don't know why, take it up with the math. The discrimination survives. It just went proprietary. It hid inside the weights where no subpoena reaches.
This has a name in the field: proxy discrimination. Strip out race and gender as inputs — legally required — and the model reconstructs them from your shopping, your neighborhood, your name's linguistic origin, your phone. It re-derives the forbidden variable from a hundred legal ones and prices you on it anyway. The bias didn't leave. It got laundered through complexity until it looked like objective math. The black box is not a flaw in the fairness project. For anyone who benefits from unaccountable sorting, the black box is the fairness escape hatch.
Our record: In the weighing of the heart the scale was public — the feather visible, the judgment spoken, the deed named aloud so it could be answered. A judgment you cannot see cannot be a judgment of Maat; it is only Sekhem, raw power, wearing the mask of order. The Shadow Neteru loves this trick above all others: it keeps the form of the weighing — the solemn verdict, the sealed door — and quietly removes the truth and the accounting from inside it. A scale that renders a sentence and swallows its own reasoning is Isfet in a clean shirt. It is not that the machine judges you harshly. It is that it judges you silently, and silence is where injustice goes to hide.
Why "we can't explain it" is a choice, not a limit
The industry likes to say explainability is technically hard. It is harder than a linear rule — true. But "hard" and "impossible" are different words, and the gap between them is full of money.
The field has real tools. SHAP values, counterfactual explanations, monotonic constraints, interpretable-by-design models that trade a sliver of accuracy for a reason you can read. A lender who wants to tell you why can get within a good approximation. The reason most don't is that a legible model is a contestable model, and a contestable decision is one you can appeal, and an appeal is friction, cost, and legal exposure. "The algorithm decided" is cheaper than a defensible reason. Opacity isn't the price of accuracy. Opacity is the product.
And notice where this is heading. The scoring engine doesn't stay at the bank. The same architecture — thousands of hidden features, one silent verdict — is being wired into insurance premiums, rental screening, hiring filters, and increasingly a fused profile that follows you across all of them. One day the number arrives before you do, and the door is already closed, and no one in the building can tell you who closed it. You were sorted by a model nobody in the room understands, defending a decision nobody in the room made.
The lever
Refuse the fog, because the black box has a fatal dependency: it only holds power as long as it stays unaccountable, and accountability is exactly what law and architecture can force.
Attack the silence with your legal right. In much of the world you already have one — the right to an explanation and to human review of an automated decision, the right to see the data used against you, the right to have errors corrected. These rights are underused because most people don't know they hold them. Use them, in writing, every time you're denied. A black box that must produce a reason on demand is a black box with a crack in it.
Attack the aggregation. The score gets its terrifying reach from fusing every stream about you into one profile. Starve it — spread your financial life across institutions that don't all feed the same broker, favor credit unions and community lenders whose logic you can actually question, and support the builders of open, portable, user-held credit — reputation you carry and reveal on your terms, rather than a dossier a stranger holds and prices on yours.
And demand the boring, radical thing: if a model decides your access to money, its logic must be inspectable by the person it judges. Not open-sourced to competitors — legible to the judged. That is not a technical impossibility. It is a design choice the powerful have declined to make, and choices can be reversed by people who refuse the default.
A verdict you cannot question is not a verdict. It is a trespass wearing a decimal point.
Make it show its work. A scale that hides its reasoning has already failed the weighing.