Session
Battling AI to Secure Public Access to Data
Everybody is scraping public data, to the point where vendors serving government entities are limiting our access. A risk to national security, if you ask me.
In this talk, I'll dive into the ongoing battles over access to public data. I run several hobby projects that have been scraping public data for years.
Now, I'm facing new challenges: rate limiting, targeted IP bans, and all sorts of other anti-scraping "security measures". These restrictions most likely stem from the surge in traffic caused by AI, either from the big names or from lots of new people creating their own scrapers that hammer the vendors.
But should "robots" really be prevented from downloading public documents?
This isn't just a theoretical question for me. One of my "robot" projects is Norske-postlister, a public service and Norway's largest public records index, listing over 180 million records. I've also sent hundreds, and in some years thousands, of freedom of information requests through the open-source project Offpost.
All of this helps both citizen and professional journalists access public information. After all, transparency and access to information are fundamental to a functioning democracy.
This is a new talk covering work and topics similar to those in my election security talk (NDC, DEFCON, Javazone, Sikkerhetsfestivalen, TDC, HelloStavanger) and also touches on my 2020 talk about Norske-postlister (Javazone).
AI provides the backdrop for this talk, but it's not really about AI. The case described in the abstract is a big part of the talk, but also serves as a bit of "clickbait" to draw attention to important subjects. It will be a civics lesson with Hallvard ("samfunnsfagstime med Hallvard"). It gets technical, with details on security measures, but anyone can follow along. Given my background in application development and security, the talk naturally leans toward a technical audience.
Hallvard Nygård
I code. I find security issues. I talk about it.
Stavanger, Norway
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top