Public figures do not make proof
French open data is plentiful and free, which is what makes it misleading. What data journalism teaches an engineer, and the step almost nobody takes.
Data journalism and data engineering handle the same files, but their results are judged by different people. In a company, a table is approved by whoever asked for it. In a newspaper, it is challenged by the reader who disagrees. That second check is more demanding, which is why an engineer has something to learn from it.
In France most of it is open and documented: the interministerial portal, the land registry, the address base, the geoportal, and datasets published directly by operators such as the railways or the electricity distribution network. The difficulty lies in joining files more than in accessing them. Two public files almost never share a granularity or a cut-off date, and that is the step where errors appear that are no longer detected afterwards.
- data.gouv.fr
- cadastre.data.gouv.fr
- adresse.data.gouv.fr
- geoportail.gouv.fr
- data.sncf.com
- Banque de France · Webstat
When the figure is not published, you have to read it where it is displayed. This is a last resort: extracting from a page depends on its current layout and stops working at the first change. It is justified for a one-off investigation, never for a lasting system. Confusing the two leads to delivering a pipeline that will break without warning.
The tooling I kept from that period is the kind that stays small: a dataset published as a database you can query without a server, or a notebook run in the browser. In both cases the article and the data are published together, and the reader can redo the calculation. A corporate data pipeline almost never allows this, and it would benefit from doing so.
In a research paper, a figure illustrates a result established elsewhere. In a news article, the figure is the result. The choice of scale, axis origin and palette then becomes an editorial choice, and a continuous palette applied to categories misleads the reader without their noticing. The tools that produce these figures are good, but the responsibility for those choices stays with the author.
My 2020 notes were half a directory of links, and I am not republishing them as they were, since a directory is what ages fastest. The order of operations still holds: find, join, verify, publish. The step I most often see skipped in companies is the third, and it is the one that allows a table to be presented as evidence.