© 2024 Peter N. M. Hansteen (2024-12-06)
Beware of robots generating your references. They could very well take it upon themselves to lie on your behalf. Testing in 2023, I observed a whopping 25% truth in the result. What is the truth rating in 2024?
Note: This is a field note that was originally intended as a fediverse (mastodon) post, but the text would have overflowed into a thread and would then have become quite cumbersome.
Back in the early days of generative Artificial Intelligence revolution (read: late 2023) I had heard enough of the hype that I did a few minimal experiments (see my earilier notes ChatGPT opines on IPv6 procastination, waxes lyrical over OpenBSD (also prettified, tracked) and I asked ChatGPT to write a pf.conf to spec, 2023-06-07 version (also prettified, tracked)) that had me conclude that for my own purposes at least, the technology as it appeared at the time was unlikely to improve productivity for myself or any of my immediate colleagues.
The TL;DR would be that since the models will only become somewhat useful if they are trained on a sufficient volume of material within the topic of interest, clearly a lot of work needed to be done. Basically, the gaps in the training data seemed to fit essentially most of the field I deal with professionally, beyond of course a few trivial generalities. And those were cracks that could only be papered over at the cost of pulping several good-sized forests. Note: This piece is also available without trackers but classic formatting only here.
Not easily discouraged, I thought maybe, it just might be, that the data the language model trainers had sucked in from public sources on the Internet would contain enough factual information that the system would be able to cough up a reasonable bibliography of what I had written.
So I opened another session with the same robot and typed
what are the most read articles by Peter N. M. Hansteen?
After a brief pause, the cursor started moving and presented its result:
As an AI language model, I don't have real-time access to up-to-date information or specific statistics on article readership. Additionally, my training only goes up until September 2021, so I may not be aware of the most recent articles published by Peter N. M. Hansteen. However, I can provide you with some notable articles that Peter N. M. Hansteen has written up until my knowledge cutoff date.
Peter N. M. Hansteen is a well-known author and has written extensively on the topic of networking and computer security, particularly focused on OpenBSD and firewalls. Some of his notable articles include:
"The Book of PF: A No-Nonsense Guide to the OpenBSD Firewall" - This is a comprehensive guide that provides practical information on setting up and configuring firewalls using OpenBSD's PF (Packet Filter) system. It covers topics such as filtering network traffic, NAT (Network Address Translation), and quality of service (QoS).
"Building Firewalls with OpenBSD and PF" - This article discusses the concepts and techniques involved in building firewalls using OpenBSD and its PF system. It provides practical examples and guidance on firewall design and implementation.
"Securing the Edge: A Practical Introduction to OpenBSD Packet Filtering" - This article focuses on the fundamentals of packet filtering using OpenBSD's PF system. It covers topics such as rule syntax, filtering criteria, and advanced filtering techniques.
"So You Want to Be a Firewall Wizard? Part 1" and "Part 2" - These two-part articles provide an introduction to firewall concepts and techniques, with a focus on OpenBSD and PF. They cover topics such as rule sets, network address translation, and handling complex firewall configurations.
Please note that the popularity of articles can vary over time, and it's always best to refer to the author's official website, blog, or other reliable sources for the most up-to-date information on their publications and readership.
(Preserved as text, cut and pasted from the website, here)
My initial reaction when I saw the result was a sound you would associate with something quite unpleasant happening.
I'll take one paragraph at the time.
Though it was probably intended as a disclaimer, the claims in the first paragraph there are, as we shall see, only 25% true, if even that.
The second paragraph has a description of me that is on par for what would turn up in a not terribly specific search using your favorite Internet search engine.
Just how well known an author and what consitutes extensive writing is open for discussion, but at least the paragraph as a whole is not counter-factual.
I did write The Book of PF. The third edition of book would have been available via that link and a few others at the September 2021 cutoff date, and the description is a valid, if somewhat short of exhaustive, description of the book when it comes to subjects covered.
However, the three titles that follow were not written by me.
I think I have read articles matching those titles at some point, but the actual authors of those pieces would not be terribly pleased to have them attributed to somebody else. I would assume their publishers would like a word there too.
I have mentioned this session a few times to colleagues and friends over refreshments, but I have not really give it much thought since.
But the chatter about how the AI bot herders are heroically fighting their charges' tendency to hallucinate seems to never come to an end.
So my question to you is: Has the situation actually improved since then?
If you have any useful experience to share or other input on the subject, please follow up with a comment in the thread where you found the reference to this note.
A reasonable followup question would be,
followed by
Come to think of it, unless the various bots that scrape the net actually obey my
At EuroBSDcon 2025, there will be a Network Management with the OpenBSD Packet Filter Toolset session, a full day tutorial starting at 2025-09-25 10:30 CET. You can register for the conference and tutorial by following the links from the conference Registration and Prices page.
Separately, pre-orders of The Book of PF, 4th edition are now open. For a little background, see the blog post Yes, The Book of PF, 4th Edition Is Coming Soon. We are hoping to have physical copies of the book available in time for the conference, and hopefully you will be able to find it in good book stores by then.
robots.txt directives, publishing even this text here will lead to further confusion over who actually wrote those titles. This is bound to turn up in the AI slop presented to some poor soul looking for valid information.
Those Titles the AI Bot Thought I Had Written is © 2024 Peter N. M. Hansteen (published 2024-12-06)
You might also be interested in reading selected pieces via That Grumpy BSD Guy: A Short Reading List (also here).
Posts
