- date written
- minutes to read
- 3
- sources cited
- 2
Why loan rates cannot be collected automatically the way deposit rates can
I checked 16 lending pages at major banks: 3 published a promotional rate with the term it applies to. The database holds 234 rows of base rate, 2 rows of margin, and 0 banks with both. How I handled it: two machine layers and one human layer.
- lãi suất 24h
- data
- operations
For deposit rates, a scraper is enough. Every bank posts a savings rate table by term, updates it regularly, and a machine can read it. After a few weeks the Lãi suất 24h database held thousands of deposit rows, each tagged with a date, a term and a bank code.
Lending rates are a different story. My first mistake was assuming the two behaved alike.
Count before you believe
On 7 August 2026 I opened 16 lending rate pages at major banks and counted by hand: only 3 published a promotional rate together with the term it applies to. The other thirteen said "attractive rates" and invited you to call.
By 14 August 2026, counting inside the database: 234 rows of base rate, 2 rows of margin, and 0 banks with both. Yet the total cost of a 20 year loan is decided from year two onward, by exactly the formula reference rate + margin. The half that decides the cost is not on any bank's website.
Those two numbers changed how I built the product: there is a layer of data a machine cannot reach, and I designed for it.
The web page is a table of contents; the PDF is the data
Second lesson: a bank's lending rate page is mostly a table of contents. The real schedule sits in the PDF it links to. One bank published the whole set, reference rate by term, the reset period, and the margin (0.5% floating, up to 3.0%), inside a PDF, while its web page left every number blank.
Reading the text on the web page and concluding "this bank does not publish it" is reading the table of contents and declaring the book empty. I wrote a tool to find PDF links inside the pages already captured: 97 links across 12 banks, all of it material already fetched and never read.
Three layers, in descending priority
The lending rate table now merges three layers:
| # | Layer | Source | Who writes it |
|---|---|---|---|
| 1 | Survey | survey file kept in the repository | A person, after asking the bank directly |
| 2 | Published | archive of captured bank pages | Machine reading web pages and PDFs |
| 3 | Market | public group posts | Machine reading Facebook groups |
The important part: the label is attached per cell. A bank can be surveyed for the 12 month column while the 24 month column still has to come from the web. A survey older than 90 days drops below the published layer and stays in the table, because a rate that was true once beats an empty cell.
Layer 1 is the human layer, and it costs less than I expected: 26 banks over 13 weeks is 2 calls a week, one full round per quarter. Results go into the survey file with the date and the channel, then into the repository history. That history is the survey record, enough on its own without adding another system.
What I took from it for other work
- Count first, conclude second. "3 out of 16" and "0 banks with both" are two numbers that cost half a day to obtain, and they were worth more than every assumption I had written into the product spec.
- Find the human layer as early as you can. Find it early and you can design a rhythm for it, 2 calls a week. Find it late and the product promises more than it holds.
- Label every cell. When data arrives from several sources of differing reliability, the reader needs to know where each cell came from, not just each table.
If you build financial products and need a rate source labelled cell by cell, tell me the context.