The intention of this prize is to encourage development of intelligent compressors/programs as a path to AGI.
At first look, it may seem that compression algorithms and AI have nothing in common, but some of the most important AI researchers focused on building AIs that used human knowledge (e.g. expert systems, knowledge bases, etc). If an algorithm can compress human knowledge itself (as opposed to words/letters), then it would appear to be doing some sort of reasoning on the input text. But the appearance of reasoning is not proof of reasoning. Read about the Chinese Room problem to see why.
It is odd that the Hutter Prize requires lossless compression. Any adult should know that that is not relevant to intelligence. Even >140 IQ people have highly lossy memories; and there are many average intelligence people who remember things nearly losslessly. Ergo, lossiness/losslessness is not relevant to intelligence.
There is another irony in the Hutter Prize. Google Search has been indexing an ever-growing chunk of human knowledge (i.e. websites), and it can return near instant search results to billions of people across the planet. Google Search is a genuine form of AI, albeit special-purpose AI. It is closer to Hutter's goal than any compression algorithm, because Google Search has to be able to recall relevant links based on error-filled user search terms. And it manages to find the right link usually within one or two attempts. That sort of memory recall is beyond anything that any human could ever do, but nobody suggests that Google Search has brought us any closer to AGI.
Why can't the Hutter Prize just be a prize to encourage the development of better compression algorithms? Why must it be wrapped up in the quixotic quest of making AGI? Why do so many allegedly intelligent people stumble over themselve to to make their work sexy?
What's more, there is a poetic irony that the website for the Hutter Prize has tons of broken internal hyperlinks. It is a website that gradually forgets irrelevant info and gains new info. The website is closer to being intelligent than all the compression algorithms made for the prize.
Thanks for this thoughtful and very interesting response.
You're very welcome. I usually write in order to solidify my thoughts, but with a small hope that someone else out there might benefit.
There is an uncountable number of unknown inter-connections between all fields of human knowledge. My favorite hobby is to search for those inter-connections.
For instance, I found that AI research is very similar to Quantum Computer research, in many ways. If you want a taste of that inter-connection, then read this and then this; and then compare the numerous successful special-purpose AI/QC projects (e.g. Google Search, D-Wave) to the numerous failed general-purpose AI/QC projects (i.e. all of them). There are probably an equal number of both.
Another inter-connection: I only discovered the AI/QC inter-connection because of my readings into Lisp, which used to be the epicenter of AI research -- and the above blog post on LZ77 is filled with Lisp code.
Kind of reminds me about the series Silicon Valley. I didn't watch the full series but I spoilered myself about the end.
That is a very relevant comparison. Per Bylund pointed out that many tech companies, especially in the real Silicon Valley, are plagued by their incessant focus on interesting but ultimately unwanted products.
A better compression algorithm is neat (e.g. in the show Silicon Valley, and in Hutter's prize), but why would I, a consumer, pay for that? Incidentally, Hutter explains that working on compresison algorithms is a sign to potential employers that the programmer is highly-skilled. But the irony doubles over on itself when such a programmer winds up at an SV tech company working on a tech project that nobody wants.
At first look, it may seem that compression algorithms and AI have nothing in common, but some of the most important AI researchers focused on building AIs that used human knowledge (e.g. expert systems, knowledge bases, etc). If an algorithm can compress human knowledge itself (as opposed to words/letters), then it would appear to be doing some sort of reasoning on the input text. But the appearance of reasoning is not proof of reasoning. Read about the Chinese Room problem to see why.
It is odd that the Hutter Prize requires lossless compression. Any adult should know that that is not relevant to intelligence. Even >140 IQ people have highly lossy memories; and there are many average intelligence people who remember things nearly losslessly. Ergo, lossiness/losslessness is not relevant to intelligence.
There is another irony in the Hutter Prize. Google Search has been indexing an ever-growing chunk of human knowledge (i.e. websites), and it can return near instant search results to billions of people across the planet. Google Search is a genuine form of AI, albeit special-purpose AI. It is closer to Hutter's goal than any compression algorithm, because Google Search has to be able to recall relevant links based on error-filled user search terms. And it manages to find the right link usually within one or two attempts. That sort of memory recall is beyond anything that any human could ever do, but nobody suggests that Google Search has brought us any closer to AGI.
Why can't the Hutter Prize just be a prize to encourage the development of better compression algorithms? Why must it be wrapped up in the quixotic quest of making AGI? Why do so many allegedly intelligent people stumble over themselve to to make their work sexy?
What's more, there is a poetic irony that the website for the Hutter Prize has tons of broken internal hyperlinks. It is a website that gradually forgets irrelevant info and gains new info. The website is closer to being intelligent than all the compression algorithms made for the prize.
Thanks for this thoughtful and very interesting response. I read all comments to @hn.
Didn't think about compression and AI in this way yet.
Kind of reminds me about the series Silicon Valley. I didn't watch the full series but I spoilered myself about the end.
You're very welcome. I usually write in order to solidify my thoughts, but with a small hope that someone else out there might benefit.
There is an uncountable number of unknown inter-connections between all fields of human knowledge. My favorite hobby is to search for those inter-connections.
For instance, I found that AI research is very similar to Quantum Computer research, in many ways. If you want a taste of that inter-connection, then read this and then this; and then compare the numerous successful special-purpose AI/QC projects (e.g. Google Search, D-Wave) to the numerous failed general-purpose AI/QC projects (i.e. all of them). There are probably an equal number of both.
But back on topic a little, here's another rabbit-hole regarding compression: Refactoring with LZ77: compression is compilation (?).
Another inter-connection: I only discovered the AI/QC inter-connection because of my readings into Lisp, which used to be the epicenter of AI research -- and the above blog post on LZ77 is filled with Lisp code.
That's good to know.
That is a very relevant comparison. Per Bylund pointed out that many tech companies, especially in the real Silicon Valley, are plagued by their incessant focus on interesting but ultimately unwanted products.
A better compression algorithm is neat (e.g. in the show Silicon Valley, and in Hutter's prize), but why would I, a consumer, pay for that? Incidentally, Hutter explains that working on compresison algorithms is a sign to potential employers that the programmer is highly-skilled. But the irony doubles over on itself when such a programmer winds up at an SV tech company working on a tech project that nobody wants.
This link was posted by kelseyfrog 56 minutes ago on HN. It received 45 points and 28 comments.