Two additional points: OA announced $1b of commitments, but did not receive that (you can check the OA Form 990s if you doubt me); GPT-2-1.5b may be 10x the size of GPT-2-124m, but it's not 'ten times more intelligent' in any sense, either in terms of benchmark scores or the fundamental accuracy of prediction which is arguably an absolute measure of general intelligence (and where the larger model would've been something like 1/3rd more intelligent).
The GPT-J mention is a lost opportunity to discuss the Latitude scandals and the multiple competitors which have sprung up and are doing interesting things to compete with AID2; even Latitude has not entirely stood still, just announcing more structured games (pixel art a la CLIP - also an OA tech - Pixelray, and sim city management). The competitors are, for example, working on going back to CYOA approaches which allow reuse of generated text, for potentially big cost savings. And AID2 has had genuine impact on research by being a GPT-3 backdoor: the first 'monologue' tests, showing that GPT-3 can be smarter when it's coached into breaking up question-answering into multiple steps, were done with AID2.
"growing damage to an institution institution"
Thanks!
Two additional points: OA announced $1b of commitments, but did not receive that (you can check the OA Form 990s if you doubt me); GPT-2-1.5b may be 10x the size of GPT-2-124m, but it's not 'ten times more intelligent' in any sense, either in terms of benchmark scores or the fundamental accuracy of prediction which is arguably an absolute measure of general intelligence (and where the larger model would've been something like 1/3rd more intelligent).
The GPT-J mention is a lost opportunity to discuss the Latitude scandals and the multiple competitors which have sprung up and are doing interesting things to compete with AID2; even Latitude has not entirely stood still, just announcing more structured games (pixel art a la CLIP - also an OA tech - Pixelray, and sim city management). The competitors are, for example, working on going back to CYOA approaches which allow reuse of generated text, for potentially big cost savings. And AID2 has had genuine impact on research by being a GPT-3 backdoor: the first 'monologue' tests, showing that GPT-3 can be smarter when it's coached into breaking up question-answering into multiple steps, were done with AID2.
Finally CYS makes an appearance 😜