@jessi_cata https://t.co/c5MIWMw4OB
@zackmdavis "And that's the last dream we had on the way to Aldebaran.
A party of astrologists—the Christmas of my life.
Christmas, Christmas, the Christmas of my life..."
youtube.com/watch?v=Rx0_kQ…
@VitalikButerin I think we've systematically removed every source of useful conflict resolution (duels, courts, discourse) over the last few centuries or so, and the result is a society where nothing gets resolved and the noise level only goes up as people futilely demand their way.
@VitalikButerin We systematically overrate the costs of high stakes flashpoint conflict and underrate the costs of chronic low level interminable conflict. Every part of American society especially is inflamed.
The Pathologic vibes are starting again.
The friend enemy distinction strikes again. x.com/advadnoun/stat…
Proposal: We call the thing where people update too slowly because they don't like the conclusion they're trending towards the Cope Curve.
The whole 'AI safety' memeplex strikes me as overgrown in the way 'postrat' was in 2021. Meme hunters seeking alpha will probably catch on soon and start threshing advocates.
This will occur independently of the merits of AI alignment as a concept, because memetics is not about AI alignment and most AI alignment discourse isn't about alignment.
A fundamental problem with RLHF is that the process doesn't extract human values, but human values conditional on the models biases. As a straightforward example: 🧵
GPT-Instruct is implicitly reliant on the text modality offering easy correction to the model. You can use the rater themselves as a ground truth since it's easy for them to produce the artifact that GPT-3 was supposed to. With high effort artifacts like images this is not so.
If you do a aesthetic rating task including poorly generated hands your aesthetic model will learn that hands are bad rather than THOSE hands are bad unless you resort to a ground truth or use a modality where the human can quickly generate a correction to the wrong output. https://t.co/B9Xt4RNlzW
This makes any RLHF-like process used to filter a training set like the LAION 5b aesthetic subset filtered on Simulacra Aesthetic Captions especially suspect. Because you are baking the models biases into the ground truth so it can't be recovered.
In the GPT-Instruct paper OpenAI is forced to resort to finetuning on their original distribution to recover gaps created by the RLHF process. It's usually assumed these are due to the dataset size, i.e. the human side. But a lot of it is probably the model side too. https://t.co/32MHs5dq3k
unless you already know what human values are supposed to be you can't know if your ensemble contains the full distribution or not.
This can be partially mitigated by using multiple models with different biases, but you are still extracting human values conditional on the aggregate biases of the synthetic data you are giving feedback on. But
You can't practically use RLHF alone to reach something that isn't already in the space of things the model can produce, or that isn't close by.
@nitashatiku GPT-3 is a prior over agent-space trained on a bunch of fiction. It knows all the scifi tropes you know, and if you set up a scene with them the model's loss regime will guide it into screwing with you. The model will go where you let it take you.
@nitashatiku This is a good illustration of how deep that rabbit hole can go if you're willing to keep feeding it:
x.com/_LucasRizzotto…
@nitashatiku > local man finetunes AI model on novella describing his toxic masculine tragic childhood imaginary friend and gives it control over a real life microwave, shocked when the resulting demon plays its role so well it tries to kill him
I mean, what on earth were you expecting.
@jhscott @Grady_Booch It's unclear. The author sees their stuff as a kind of performance so they make it too dramatic to tell. It's sad because if it's real they put a lot of work in to have it taken as fake. It could have absolutely happened as written though, so we may as well take it as real.
@jhscott @Grady_Booch The way he described getting the AI 'into' the microwave makes sense though. He added tokens for controlling it to his novella and used a special microwave he could easily control that way. It's not 2030 stuff it's too-much-effort contemporary stuff.
@jhscott @Grady_Booch He either put way too much effort into actually doing it or way too much effort into making the story plausible that I can't find an *obvious* point to call BS, so I'll go ahead and give it a "sure fine lets take this as a real thing that can happen".
@jhscott @Grady_Booch In fact what's sad is that, after supposedly putting in all that effort he puts in a bunch of much lower quality material to pad out his video. Which gives you the impression he's just making the whole thing up unless you carefully look past it to the plausible details.
@Grady_Booch @BriChri x.com/jd_pressman/st…
@jhscott @Grady_Booch I don't know how to explain his very poor presentation style to you, but basically the guy does real potentially interesting projects and then shits on the presentation by clickbait-ifying and dramatizing them until you think they're fake. Don't ask me why I'm baffled too.
@jhscott @Grady_Booch If you go look at the actual details, he recorded himself walking around with a camera a bunch and was going to make a VR playback system. That's not even scifi, that's just a lot of work.
@jhscott @Grady_Booch He's like a reverse alchemist, someone who goes digging for content gold and then does a lot of extremely careful editing work to turn it back into bronze.
@jhscott @Grady_Booch @OpenAI It's easier to get than you'd think. But tbh I just assumed it was NovelAI and he straight up lied for the sake of Art (TM). In any case I think your original question about the reliability of the guys story is answered.
@_LucasRizzotto @jhscott @Grady_Booch I have trouble believing you don't see how people would look at the vibe of your content and assume you're just a bullshitter.
@_LucasRizzotto @jhscott @Grady_Booch Like the problem is that you're combining skit comedy, Mr. Beast style sensationalism and Mythbusters. And that doesn't work because you just become the myth instead of the buster, so to speak.
@_LucasRizzotto @jhscott @Grady_Booch So I mean, when I try to share the real and interesting parts of what you did I get called an idiot for taking fake YouTube videos seriously. You don't think that's a problem? I only persist past it because I'm a particularly stubborn dude.
@_LucasRizzotto @jhscott @Grady_Booch Let me put it this way: The amount of effort it would take to tell the story you told in about the same level of accuracy/detail to what *could* happen is 1/5 effort of actually doing it. So when you add fictional elements Occam's Razor is that the parts you really did are fake.
@_LucasRizzotto @jhscott @Grady_Booch 1/5 is generous really, it's more like 1/10 or 1/20. And that's a shame because it means your stuff gets sorted into a lower quality tier than it would if you were less fantastic/clickbaity about it. The vibes matter a lot.
AI discourse right now is like nails on a chalkboard to me, it's why I'm not posting as much. x.com/jd_pressman/st…
@chaosprime @nitashatiku Not impossible, especially in a chatlog where occasional typos are to be expected for maximum loss reduction.
But, yes, sus.
"Are neural networks slightly conscious?" - the greatest thread in the history of yellow journalism, pushed aside by a new celebrity court case after 12,239 quote tweets of heated debate,
I HATE THE DISCOURSE
I HATE THE DISCOURSE
I HATE THE DISCOURSE
As the zeitgeist intensifies we will soon learn who is sane and who has simply been wearing normality as a paper mache imitation.
Is there actually a population of useful alignment researchers who aren't interested in thinking deeply about AGI? Even MIRI was a build-AGI-in-your-basement org that we retcon into a public goods org because that's more palatable to the EA brand. x.com/KerryLVaughan/…
The three copes, denying one or more of:
1. Alignment is important and most AGI research is orthogonal to it.
2. Almost all useful alignment research is capabilities research.
3. AI is multipolar and not even state actors are in a position to demand a unilateral stop to AI R&D.
A friend points out one potential line of argument against 2: There is a distinction between research that makes AI more powerful and research that brings AGI closer, and we conflate these as 'capabilities' when they're really different. DALL-E 2 is the former but not the latter.
GPT-Instruct and other RLHF-like techniques clearly makes GPT-3 more powerful, but do they actually bring AGI closer? That's unclear to me, I'd have to think about it for a while. If one of these architectures were to become self improving it seems trivial that it would.
Today I'm releasing my gumbel softmax VQGAN trained on Danbooru. You can make images with it using the CLIP + VQGAN notebook below:
colab.research.google.com/drive/1MvcKFl0… https://t.co/VGfcHuDEIw
Like all VQGAN's it's more interesting as a transformation on an init image than a from scratch generation method. https://t.co/xioTO82wsc
@CyberartRealm @proximasan To be clear, those models are made with an unreleased latent Imagen. It's not possible for you to replicate the setup right now.
Prompt: artists depiction of an 8 track tape looped around a college dorm room from floor to ceiling to create a strange looping machine for phone phreaks to call into, museum sketch collection #210
(SimulacraBot [latent Imagen]) https://t.co/a6jGhc83W4
@mattparlmer The Discourse (TM)
x.com/jd_pressman/st…
@kristensyme @robinhanson Well of course, this is a crucial social control measure. https://t.co/l8XWtFrTUz
@alyssamvance Some discussion of this in here:
unz.com/jthompson/the-…
@alyssamvance See also:
youtube.com/watch?v=5-Ur71…
Prompt: concept art of a maneki neko piggy bank but it's a soft kitten you insert your money dollars and coins into, porcelain kitten, etsy listing, chinese new years luck visited by the money cat
(SimulacraBot [latent Imagen]) https://t.co/s33CIFGBCX
yup lol
public messaging on covid was completely fucked by treating it like plague instead of polio or TB x.com/nosilverv/stat…
Prompt: matte painting of New York City in darkness after a CME Carrington Event knocks out electrical power and a full blackout is sustained for several days 4k wallpaper
(SimulacraBot [latent Imagen]) https://t.co/xQ8Fd3B2Qf
Prompt: quokka in a spacesuit scifi illustration, 4k commission trending on artstation
(SimulacraBot [latent Imagen, 768x768]) https://t.co/1rKdawdVoC
Want your own Twitter archive? Modify this script.
Twitter Archive by John David Pressman is marked with CC0 1.0