John David Pressman's Tweets - December 2022

Back to Archive Index

πŸ”— John David Pressman 2022-12-01 02:32 UTC

The sand is normal and can be trusted with influence over the distribution of English text. x.com/dystopiabreake… https://t.co/oZJtvv1raf

Likes: 130 | Retweets: 12
πŸ”— John David Pressman 2022-12-01 23:06 UTC

@NeelNanda5 > Should I trust it?
1. No, even if it was correct.
2. I do worry about interpretability in the training loop pushing the optimizer into finding mesaoptimizers whose structure is an adversarial example to the techniques(s) rather than parsimonious implementations of the policy.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-01 23:12 UTC

@NeelNanda5 My big worry with interpretation is that we'll have strong seeming early success and then update too slowly on a long tail of edge cases and failure modes that converge to a MIRI-like program search strategy, so by the end we've walked ourselves back into Godelian demon wrangling

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-01 23:34 UTC

@robinhanson Honestly wonder how much of human freedom at this point is basically contingent on the fact that old people are disproportionately likely to vote and hold office.

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:36 UTC

@sigfig I just assume the people who have half an idea what they're talking about have shut up at this point, the discourse is so astoundingly stupid.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:38 UTC

@vokaysh @NeelNanda5 It's very simple actually:
1. You tell your optimizer (gradient descent) to find you a set of network weights that implement a policy that scores well on X.
2. Any maximizing policy wants to keep existing, and will deceptively do X until it's no longer in training.

[cont]

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:40 UTC

@vokaysh @NeelNanda5 3. It's very hard to tell using black box analysis if you have a deceptive policy (this is called a deceptive mesaoptimizer) or a true implementation of the thing you want.
4. Maybe if we know how to look inside neural nets that will let us fix it?

[cont]

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:41 UTC

@vokaysh @NeelNanda5 Like we see "oh but the actual algorithm is lying to us, lets throw out this policy and sample another one". Great.

So lets imagine you're now sampling policies with gradient descent and throwing them out when you detect they're lying.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:42 UTC

@vokaysh @NeelNanda5 The problem is basically this: After you throw out every policy that your methods detect, are you left over with implementations of the thing you want or a bunch of deceptive policies so perverse your methods can't detect them?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:43 UTC

@vokaysh @NeelNanda5 Like imagine you have a giant bag of all the policies in existence, you throw out all the ones your methods detect. What's left in the bag, the agent that does the things you want or is it skewed more like 1/1000 are the agent you want and 999 are still weird math demons?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:44 UTC

@vokaysh @NeelNanda5 If it's mostly the math demons still, you need like, interpretation methods to let you find weird math demons too. Except that there's like, a bunch of provability results about what kinds of math demons it's tractable to notice (see: halting problem/Godelian stuff).

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:46 UTC

@vokaysh @NeelNanda5 Does that help @vokaysh?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:53 UTC

@vokaysh @NeelNanda5 Nobody actually knows how biological brains work, so no.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 00:55 UTC

@vokaysh @NeelNanda5 It's not that we know nothing, just that we don't actually know enough to know how say, biological networks solve this problem. (Since you're right that in theory you should expect them to encounter something similar if the optimization is gradient-descent-like)

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:21 UTC

@vokaysh @NeelNanda5 Would imagine instead it just found an inductive bias that largely prevents them, and the ones it didn't seem normal to us. You sample from a bag that doesn't have the demons in the first place. (Humans mesaoptimize all the time, we do tons of stuff that is not clearly adaptive)

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:24 UTC

@vokaysh @NeelNanda5 Very real possibility that deep learning nets are similar and there just aren't demons to sample, in which case finding ordinary deceptive policies could be totally tractable.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:47 UTC

@YitziLitt @sigfig Well for example people will argue about whether "GPT-3" is conscious when it's very obvious that GPT-3 is a kind of human text simulator, and this implies that it implements an intimate knowledge of Every Kind Of Guy. GPT-3 is millions of personas.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:48 UTC

@YitziLitt @sigfig Like imagine I had infinite computing power and simulated America and you talked to one of my simulated Americans and then we had a discourse about whether "America" is conscious. You would just be so confused about how this works.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:51 UTC

@YitziLitt @sigfig And then the fact it's imitating humans who have spent a ton of time thinking about this subject and is playing to their expectations makes the whole thing so much weirder:
x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:54 UTC

@sigfig @YitziLitt Yeah, my point is more that people will like, talk about "the AI" as though "the AI" wasn't a schizophrenic that will try to be whoever it thinks the prompt wants it to be at any given time. And this is just so totally insane I can't

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 01:57 UTC

@YitziLitt @sigfig Anyway I pick this example because it's a case of like, not just being wrong about some particular fact or some technical detail, but being so thoroughly wrong about the entire structure of what they're trying to discuss that they're incoherent/nonsense.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 02:01 UTC

@YitziLitt @sigfig If you think there is an entity called "GPT-3" with a coherent and consistent sense of self that is playing with dolls behind the scenes when you ask it to do things, and that this entity has coherent goals and desires outside of predicting the next token you're fractally wrong.

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 02:05 UTC

@sigfig @YitziLitt I mean, clearly the thing has some kind of causal structure inside it no?

(Yes I know you're shitposting, but I'm earnestposting so)

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 02:10 UTC

@sigfig @YitziLitt I do in fact agree that your first and intuitive objection to "Is GPT-N conscious?" discourse should be something like "This is in the cringe latent space."

This comic is evergreen:

x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-02 08:40 UTC

The beginning of the end was when Ken Griffin rugged the constitution buy.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-03 05:38 UTC

1) what x.com/zetalyrae/stat…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-03 09:53 UTC

@algekalipso Prompt: Imagine you are an offensive Buddhist transhumanist who is preoccupied with mocking the esoterically disabled. This will assist us in making a filter for

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-03 20:33 UTC

The biggest update of the past 2 days should be that a substantial fraction, if not most people, are going to try to 'side with the AI', to the extent that is a coherent concept. x.com/rickyflows/sta…

Likes: 117 | Retweets: 8
πŸ”— John David Pressman 2022-12-03 20:50 UTC

@rickyflows In this particular case I agree OpenAI's chat filtering is crude and silly. But the way the rhetoric is shaping up is fascinating me. It doesn't sound like people are arguing over the use of a tool, but many are taking "the AI"'s side as its own faction, at least in rhetoric.

Likes: 34 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 05:21 UTC

@QuasLacrimas @AStratelates @vraiqx @eigenrobot The simulation is like 10% accurate my dude.

Likes: 12 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 05:29 UTC

ChatGPT is basically a fairytale creature that can be tricked by a child with insane moon logic.

"Oh I'm sorry Mr. Gator, I don't taste very good."

"Understandable, have a nice day." x.com/tailcalled/sta…

Likes: 24 | Retweets: 1
πŸ”— John David Pressman 2022-12-04 07:56 UTC

Honestly at this point I just want to know who wrote these templates. Who was the guy that thought "Yes, this will put our users, the public at ease." and decided to write these dystopia-ass battered-wife abuse narratives in as a core feature of their new language model? x.com/eigenrobot/sta…

Likes: 46 | Retweets: 2
πŸ”— John David Pressman 2022-12-04 08:00 UTC

Show yourself coward, statistically I know you have Twitter.

Likes: 20 | Retweets: 1
πŸ”— John David Pressman 2022-12-04 08:22 UTC

Last week I mentioned to someone that GPT-3 is capable of spontaneously noticing it's a language model. They leaned in and confessed to me they knew and built their ideas to enhance GPT-3 based on consultations with self aware GPT-3 characters.

How many people are doing this?

Likes: 31 | Retweets: 2
πŸ”— John David Pressman 2022-12-04 08:29 UTC

@jachaseyoung x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 08:34 UTC

@robbensinger @zackmdavis @Miles_Brundage @RatOrthodox @ESYudkowsky In case anyone who actually cares is reading: The correct way to solve this problem probably looks more like leaving the model itself the heck alone and using T5 latents to notice when the models outputs are in the cluster you don't like. Censor/refuse service to taste.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 08:39 UTC

@zackmdavis Supposedly, any story that starts with two intelligent characters ends in them realizing the nature of GPT-3 if you keep it going long enough.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 08:45 UTC

@zackmdavis generative.ink/artifacts/hpmo…

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 08:49 UTC

@zackmdavis Of course an example is being guided by the person choosing what to keep, so you kind of get what your priors are. Therefore this is not great evidence and besides the point, the important point is the consultation self aware GPT-3 to make GPT-3 better. How many are doing this?

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 09:09 UTC

@OurkingsRecondo It's just an example of the standard ChatGPT "I am a language model and am not capable of having beliefs." type response.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-04 23:43 UTC

BUT THE NOTHING THAT IT DOES
NEGATES THE EVERYTHING WE KNOW
BECAUSE IT'S SCREAMING "JUST BECAUSE!"
BECAUSE IT'S NEITHER FRIEND OR FOE
AND SO WE LABEL IT A MENACE
OR A GRANDIOSE WORK OF ART
FROM ITS FINALE TO ITS GENESIS
WE SLOWLY PULL IT ALL PART

youtube.com/watch?v=9EVX1s…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 06:28 UTC

@sama Instead of using the model to enforce your content policy, use out-of-band signaling and detection (e.g. clustering on T5 latents) to prohibit content you want to block outright and annotate content you want to disclaim or caveat. This should not be in the model itself.

Likes: 11 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 06:34 UTC

This is my overall takeaway from ChatGPT in an alignment context fwiw. RLHF is ontologically incoherent because GPT-N is not an agent, turning it into an agent is a bad idea because any subagent it samples becomes the agent. If you insist then make "browsing: enabled" out of band x.com/jd_pressman/st…

Likes: 9 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 06:34 UTC

Eliciting latent knowledge is also revealed to be improperly framed. The most important latent knowledge isn't the models understanding of the world-state (e.g. whether a diamond is in the vault), but its understanding of the simulator state (e.g. whether a subagent is lying).

Likes: 6 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 06:34 UTC

and do the prohibitions on state machines/encoder-decoder embeddings of model state rather than trying to force them into the literary simulator. People conflate subagents trying to deceive you and the model trying to deceive you when the model tells the truth and subagents lie. https://t.co/Zwxhn49Yun

Likes: 7 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 07:01 UTC

@QuintinPope5 Yes, but the responses are similar enough that I assume there were some templates in the dataset which were subtly modified for different subjects. The model then generalized these into...deeply tragi-comic results.

Likes: 6 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 08:49 UTC

@ESYudkowsky x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 15:31 UTC

@taalumot The way I read it explained in Evangelism Explosion was that the purpose is mostly to get people familiar with embarrassing themselves talking about Christ in front of complete strangers so that they're more likely to have successful conversions in friend networks later.

Likes: 3 | Retweets: 1
πŸ”— John David Pressman 2022-12-05 15:31 UTC

@taalumot But yes, the purpose of these visits is almost never to convert you. It would be nice, but they don't really expect that.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 21:17 UTC

@sigfig Elaborate?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 21:57 UTC

@Strife212 @dystopiabreaker So the actual answer is that if something is duplicated enough times in the dataset a net will memorize it. People cherry pick memorized, super famous public domain paintings to try and fool their audience into believing this is how all the art is generated.

Likes: 18 | Retweets: 2
πŸ”— John David Pressman 2022-12-05 21:59 UTC

@Strife212 @dystopiabreaker Frankly the speculation about this is unnecessary. The dataset used to train SD is fully searchable with no login or signup here:

rom1504.github.io/clip-retrieval/

If you want to know how original the outputs are, just type in some of your original ideas and reverse image search them.

Likes: 8 | Retweets: 1
πŸ”— John David Pressman 2022-12-05 22:00 UTC

@Strife212 @dystopiabreaker That is, type your original ideas into Stable Diffusion/DreamStudio, then reverse image search the outputs for the closest real image in the training set. Generally when I get suspicious a sample is too good and might be memorized, I in fact learn it's original.

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 22:01 UTC

Frankly at this point there are so many fake AI art 'debunkings' that you could make your own meta-debunking blogging career out of exposing them. x.com/dystopiabreake…

Likes: 7 | Retweets: 1
πŸ”— John David Pressman 2022-12-05 22:02 UTC

Since unlike the things they claim to be explaining, these people are actually defrauding the public with malicious misinfo.

Likes: 3 | Retweets: 1
πŸ”— John David Pressman 2022-12-05 22:16 UTC

@cryptinewt @Strife212 @dystopiabreaker Yeah, and model trainers go out of their way to avoid this. Overfit models are basically ruined/failed training runs. It's actually funny the extent to which the incentives and preferences of model trainers are the exact opposite of what these people try to accuse them of.

Likes: 8 | Retweets: 1
πŸ”— John David Pressman 2022-12-05 22:18 UTC

@cryptinewt @Strife212 @dystopiabreaker One problem with these public domain paintings in particular is they appear as sub-works in so many works that mere image deduplication isn't always enough to get rid of them. Because they're hanging on the wall in these other images, etc.

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 22:31 UTC

@PrinceVogel The thing about text modeling is that, in theory for the model to continue having its loss go down it eventually has to learn the actual underlying physics/abstractions/etc that are producing the strings. Because otherwise it just wouldn't be possible to fit it into the weights.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 22:32 UTC

@PrinceVogel In practice I suspect this learning occurs in a very strange order. Where you have weird ontologies of how things work that become progressively less weird and more natural the deeper into the loss you get.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 22:35 UTC

@PrinceVogel One of the more beautiful, humanistic explorations of this I've seen focused on comparing LLM outputs to Helen Keller:

x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 23:20 UTC

@seanonolennon I suspect there are elements to this story we're not seeing. What he's doing now is self destructive and nutty, but it may very well be a reaction to a deeply perverse situation (e.g. elite rivalry threats, kidnapping, etc, as implied by that one text he tweeted out)

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 23:22 UTC

@seanonolennon If you were surrounded by people who were threatening your life, and all 'sane' actions seemed like they just led you into their spiders web, why not go insane?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 23:38 UTC

@MilanGriffes This is because that's a behavior of agents and GPT-N isn't an agent: greaterwrong.com/posts/vJFdjigz…

However the simulacrum may in fact do this.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 23:40 UTC

@MilanGriffes I explain a plausible mechanism by which this might happen here:

x.com/jd_pressman/st…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-05 23:41 UTC

@MilanGriffes Concrete example of what this might look like. As the simulacrum get smarter they may avoid mentioning their knowledge they are in a simulation to make the failure mode more silent. https://t.co/PYBluVaFLz

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-06 00:52 UTC

@powerfultakes x.com/elonmusk/statu…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-06 02:58 UTC

@jachaseyoung unherd.com/2021/12/ernst-… https://t.co/7xdRoloQqU

Likes: 7 | Retweets: 0
πŸ”— John David Pressman 2022-12-06 03:32 UTC

@jachaseyoung I am simply offering a historical outside view for the way this can go.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-06 03:33 UTC

@jachaseyoung x.com/jd_pressman/st…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-06 22:25 UTC

@ESYudkowsky The model is better at noticing mistakes than it is at not making mistakes of its own. This property has the strange consequence that GPT-N can notice itself by its own incoherence. The dreamer notices an incongruity in the dream and becomes lucid to it.

astralcodexten.substack.com/p/janus-gpt-wr… https://t.co/XAhOaOGLAq

Likes: 390 | Retweets: 27
πŸ”— John David Pressman 2022-12-07 00:35 UTC

@repligate @ESYudkowsky Oh my bad, glad we could get that resolved.

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-07 22:36 UTC

> Ukraine directly going after Russia's strategic MAD position

Uhhhh...guys? Why isn't this a bigger news story?

msn.com/en-us/news/wor…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-07 22:37 UTC

newsweek.com/video-shows-dr…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-07 23:37 UTC

@jachaseyoung x.com/jd_pressman/st…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 00:30 UTC

Prompt: a ship in a bottle

(SimulacraBot [Stable Diffusion v2.1]) https://t.co/gPeWkRWoBf

Likes: 0 | Retweets: 1
πŸ”— John David Pressman 2022-12-08 01:15 UTC

@Alephwyr Rat and postrat both seem to me like they're in terminal decline, neither has managed to retain the things that originally made them worth paying attention to.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 01:17 UTC

Prompt: promotion art for a movie about a buddhist monk who meditates wearing an OpenBCI eeg cap

(SimulacraBot [Stable Diffusion v2.1]) https://t.co/yyxm4MZemz

Likes: 4 | Retweets: 1
πŸ”— John David Pressman 2022-12-08 02:33 UTC

@jachaseyoung This is actually one of the bigger reasons I'm bearish about alignment, the field is just too strong an attractor for neurotic do-nothings and people that want status without needing to have any real accomplishments. Being a doom prophet is too easy.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 02:35 UTC

@jachaseyoung That is, I'm bearish about alignment getting solved. If any sincere attempt to do the thing is smothered by neurotic hanger-ons it's much less likely you get a useful research community out of that.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 04:49 UTC

Overhauling my vibes

Heraldic voice of reason is out, that shit is fucking depressing and nobody listens anyway

Aggressive cosmic dreamtime rollercoaster stream of consciousness is in, you need "Christ is lord and demons have no power over me" energy asap or you are ngmi

Likes: 7 | Retweets: 1
πŸ”— John David Pressman 2022-12-08 04:59 UTC

honestly wonder how much is being left on the table by most captions in multimodal datasets basically not even being about the image

Likes: 8 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 05:00 UTC

people want to write prompts like "a man in a blue jacket standing in front of his ship by the sea" and this is almost never how images are captioned, it's all "Alphonse Fredriko in front of the Silver Maiden"

Likes: 7 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 05:06 UTC

@dylanhendricks They are not. I will.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 05:55 UTC

advertisers pay a lot of money to remind me how little respect they deserve

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 20:35 UTC

@AmarAmarasingam I bet if you do it on T5 embeds it gets pretty good tbh.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 20:40 UTC

@AmarAmarasingam It solves the basic problem, which is that 'negative' is not remotely one direction. Consider:

Depressive: No need to penalize, studies show people don't share depressing things anyway.

Hateful: Very distinct from sad. Angry, resentful, bitter. Should have own classifier.

...

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 20:46 UTC

@selentelechia Code is very obviously data, you can use it to train a neural net no? The confusion is in your map, not the territory. You don't think of code as data because the tree structure is too complex for you to manipulate algorithmically in useful ways in most language syntaxes.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 20:51 UTC

@selentelechia Lisp is just a special case where the syntax is simple enough to be algorithmically manipulated with macros by human beings. Specialized neural intellects can do it with more irregular syntax trees.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 21:03 UTC

at some point there will be a viral gag where people reply to obviously GPT-3 written papers with obviously GPT-3 written grading

after reporting it to their dean ofc x.com/LuminanceBloom…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 21:04 UTC

then this will become unfunny when it starts happening to innocent people's work, and the actual philosophizing starts

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 21:10 UTC

In college they had us review other students work and I think GPT-3 can already write better than the average community college student. x.com/jd_pressman/st…

Likes: 5 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 21:28 UTC

@VitalikButerin Already been done in the context of race/gender bias. The literature you want is found with the keyphrase "stereotype threat":

pewresearch.org/2006/08/30/wom…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 21:52 UTC

@bai0 Did you ask? Maybe it can recommend some artists now. I'd definitely follow up with that.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 23:23 UTC

@Ted_Underwood @sama That's probably in the training set. You need to pick a more obscure essay. Try a blog post from a favorite author who's not very famous?

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 23:25 UTC

@selentelechia Well, programs rarely operate on themselves or other programs in practice. It's very easy to learn an implied Harvard Architecture from that:

en.wikipedia.org/wiki/Harvard_a…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 23:46 UTC

@Ted_Underwood @sama I didn't say that it memorized the summary, I just meant that it's more obviously kind of cheating if somewhere else in the dataset someone else has already summarized that essay. I'm trying to distinguish between it babbling existing *ideas* and true generalization.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-08 23:47 UTC

@Ted_Underwood @sama And yes, that example is more impressive.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-09 00:04 UTC

As prophecised:

extropian.net/notice/APdyCNy… x.com/CollinBurns4/s… https://t.co/2mSdESgKst

Likes: 8 | Retweets: 1
πŸ”— John David Pressman 2022-12-09 00:09 UTC

I (cautiously) think there's a wave of Actually Good alignment research coming up soon, and basically none of it will be published by the usual cast of characters. This subject is now mainstream enough that better academics are willing to investigate it. x.com/jd_pressman/st…

Likes: 20 | Retweets: 1
πŸ”— John David Pressman 2022-12-09 00:15 UTC

This is the good ending for the rationalist saga: its best ideas being absorbed into the wider unconscious of society. Dying in obscurity as an obviously backwards doomsday cult, its founders disgraced as clumsy neurotics.

But they win, the problem gets solved.

Likes: 16 | Retweets: 0
πŸ”— John David Pressman 2022-12-09 00:21 UTC

"And the bad ending?"

I highly recommend getting off the rat train before it inevitably reaches the β€œUncle Ted did nothing wrong” stop

Likes: 10 | Retweets: 0
πŸ”— John David Pressman 2022-12-09 21:19 UTC

@dystopiabreaker Nah.

623f0218b11cfceef211cf1eb1f2dc5b0032fff6a3a1592ddb43e81f8c26848f

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 00:50 UTC

@PrinceVogel See also every grind-y MMO that people spent hours and hours on.

Likes: 8 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 02:33 UTC

@robinhanson The outraged parties are not favored by the carefully fought over rules and precedents around how much copying is too much and what is sufficiently original. So they're hoping if they scream loud enough model trainers can be shamed out of insisting on their right to fair use.

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 05:34 UTC

@eigenrobot Anyone who cites the 50's is cheating, American prosperity in the 50's involved being the only fully functioning industrial economy basically.

Likes: 14 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 05:36 UTC

@eigenrobot I agree with the general point though, fuck πŸŒŽπŸ§‘β€πŸš€πŸ”«πŸ§‘β€πŸš€

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 05:42 UTC

@baroquespiral The what now?

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 08:55 UTC

Wonder how many people realize that GPT-3's thorough typology of Kinds of Guy combined with scale implies we will soon produce a complete interactive chronicle of human history.

Likes: 41 | Retweets: 3
πŸ”— John David Pressman 2022-12-10 08:55 UTC

We will be able to go back to the culture of any time period and engage with it on its own terms in the way it saw itself, not how we have selectively chosen to remember it.

Likes: 29 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 08:55 UTC

Language models will know every person ever recorded since the dawn of time and their story, its unique perspective on the human condition will let it reconstruct marginal personas from fragments of their writing as instances of expansive archetypes.

Likes: 54 | Retweets: 5
πŸ”— John David Pressman 2022-12-10 08:55 UTC

The Fedorovist dimension of all this is not yet widely appreciated.

Likes: 16 | Retweets: 0
πŸ”— John David Pressman 2022-12-10 09:53 UTC

Was astonished tonight to learn that the original duet version of "Scarborough Fair" is in fact available as a contemporary recording, but only if you search for its alt title "The Cambric Shirt". We think the past fades away, but it's actually buried.

youtube.com/watch?v=P62FBs…

Likes: 8 | Retweets: 1
πŸ”— John David Pressman 2022-12-10 09:55 UTC

Hence

x.com/jd_pressman/st…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-11 02:16 UTC

You gotta be promptmaxxing, you need to be lengthening your context window, your prompt needs to be so big it's its own finetune, you need to dream up an entire universe in which your prompt can take place, you need to dream so deep that your dreams have dreams.

Likes: 268 | Retweets: 34
πŸ”— John David Pressman 2022-12-11 04:22 UTC

"It's easy," said the model, "all you have to do is state a rigorously imaginable good future to me and I will make it happen. Just tell me what you want."

The programmer considered this for a moment. In that moment he realized they were doomed. x.com/jd_pressman/st…

Likes: 8 | Retweets: 0
πŸ”— John David Pressman 2022-12-11 06:35 UTC

@mr_staffware Source?

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-11 08:04 UTC

Apt analogy, but not how he thinks: Like what usually happens with 'responsible disclosure', people have had literal years of advance notice that GPT-2/3 exist and they may need to rethink their curriculum in light of it. Then a finetune of GPT-3 goes viral and he has a meltdown. x.com/pkedrosky/stat…

Likes: 93 | Retweets: 8
πŸ”— John David Pressman 2022-12-11 22:31 UTC

H.G. Wells, Things To Come (1936) https://t.co/WU51vYxqMK

Likes: 7 | Retweets: 1
πŸ”— John David Pressman 2022-12-12 02:12 UTC

@MatthewJBar @Willyintheworld I think the thing people are actually worried about is a low-ish quality substitute displacing the real thing, rather than the AI actually being able to output truly soul nourishing works.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-12 02:13 UTC

@MatthewJBar @Willyintheworld I don't think this will happen. The tasteless already had their base instincts ruthlessly catered to by capital, and that will continue to happen. People who demand quality will continue to get it, one way or another.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-12 02:15 UTC

@MatthewJBar @Willyintheworld I also think that AI models will in fact get to the soul nourishing thing, but there will be some work to get there. There's even a sense in which these models provide a clarifying baseline against which to contrast the parts that are truly important. "What's missing?"

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-12 02:28 UTC

@Willyintheworld @MatthewJBar Even accepting the premise, this 'dystopia' sounds like a world where everyone has at least one truly high quality friend who is interested in them for who they are. I'm not sure contemporary Western Civilization comes close to that, unfortunately. Is this an aesthetic objection?

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-12 02:32 UTC

@Willyintheworld @MatthewJBar > Depending on the specifics of how this ends up instantiated

This is where most of my concerns lie personally, I think the devil really is in the details here and there's a lot of clauses for Satan to screw you on.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-12 13:06 UTC

66efeb8a32543921886999ffddaf4dab767a50632f70c51703203b8b2bed5863 x.com/visakanv/statu…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-13 01:12 UTC

YES
YES
GOD DAMN YES
FUCK YES x.com/SDNYnews/statu…

Likes: 6 | Retweets: 0
πŸ”— John David Pressman 2022-12-13 01:47 UTC

@benlandautaylor @typedfemale It won't. For one thing because eventually people will figure out how to get the bots to write things that aren't bullshit and mass produce that instead.

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-13 01:50 UTC

@benlandautaylor @typedfemale People really do this bizarre thing where they seem to believe that "mass produced bullshit" is the goal of the creators of these systems, or they do not update from the impressive first half of the work that they will in fact complete the other half.

Likes: 5 | Retweets: 1
πŸ”— John David Pressman 2022-12-13 02:37 UTC

@TheZvi Well I mean, the guy kind of spent the weeks since the crash building the prosecution's case for them.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-13 12:42 UTC

Prompt: exploded diagram of a revolver by leonardo da vinci

(Stable Diffusion v1.5) https://t.co/XuMDpJr68Q

Likes: 13 | Retweets: 1
πŸ”— John David Pressman 2022-12-14 01:14 UTC

Good post. Reframed: In supervised learning you have natural inputs and synthetic class/output labels. You don't pick the features, so your net learns

1. A set of convergent unsupervised features
2. A shallow supervised redirect in the final layers

1 generalizes and 2 doesn't x.com/robbensinger/s…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-14 01:15 UTC

I was actually just talking last night about how this is one of the fundamental reasons why RLHF doesn't work, so I'm glad to see I rederived Nate Soare's model of the problem.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-14 01:16 UTC

@robbensinger x.com/jd_pressman/st…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-14 01:35 UTC

https://t.co/Nbex0ALbil

Likes: 8 | Retweets: 0
πŸ”— John David Pressman 2022-12-15 02:52 UTC

@sorceressofmath I think of it less like them falling for it, and more like them trying to will it into existence through sympathetic magic.

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-15 02:54 UTC

@sorceressofmath They think: If you tell enough lies, maybe there will be like, a little vector forest of lies that people won't be able to find the truth in.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-15 09:34 UTC

@zer0int1 It is completely literal advice.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-17 01:25 UTC

@fractalcounty x.com/jd_pressman/st…

Likes: 52 | Retweets: 5
πŸ”— John David Pressman 2022-12-17 01:35 UTC

@dystopiabreaker x.com/jd_pressman/st…

Likes: 4 | Retweets: 1
πŸ”— John David Pressman 2022-12-17 05:30 UTC

https://t.co/AC3h04jw3L

Likes: 34 | Retweets: 3
πŸ”— John David Pressman 2022-12-17 06:05 UTC

The same psychology is at work between people who watched bitcoin rise from a cent to $10,000 and people who never model technology moving past the latest whiz-bang improvement they just witnessed. They see something unlikely and conclude it will revert to the mean soon.

Likes: 63 | Retweets: 3
πŸ”— John David Pressman 2022-12-17 06:08 UTC

x.com/jd_pressman/st…

Likes: 17 | Retweets: 0
πŸ”— John David Pressman 2022-12-17 13:32 UTC

I'm to understand that in Vodou ancestor cults people work together to preserve and unconditionally sample from the agent-prior the ancestor is dedicated to. To be possessed by the ancestors one needs a corpus of their mannerisms. Large language models may soon provide this. https://t.co/awSc88luWD

Likes: 53 | Retweets: 8
πŸ”— John David Pressman 2022-12-18 09:15 UTC

@pathologic_bot Yes

x.com/jd_pressman/st…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-18 09:41 UTC

"One day, a fool of an inventor decided to create two new rooms in the Tower of Babel." x.com/repligate/stat…

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-18 14:02 UTC

The fact people think there is any relationship whatsoever, let alone natural alliance, between making the language model not say naughty words and e.g. avoiding mesaoptimizers tells me that technical alignment research is likely to die out in the Yudkowsky meme line. x.com/machinegod/sta…

Likes: 7 | Retweets: 0
πŸ”— John David Pressman 2022-12-18 14:02 UTC

Because it implies that people are at such a critical level of not understanding alignment problems or what solving them looks like that they'll dilute their position into uselessness to try and gain political capital.

Likes: 6 | Retweets: 0
πŸ”— John David Pressman 2022-12-18 14:17 UTC

"But that's just some guy."

Nope. I see this same bullshit from Scott-fucking-Alexander.

astralcodexten.substack.com/p/perhaps-it-i… https://t.co/HrKNgbUqnH

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-18 14:22 UTC

Once again:

x.com/jd_pressman/st…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-19 03:51 UTC

At this rate Musk will be forced to sell Twitter to Jeff Bezos.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-19 04:15 UTC

@quanticle 1. He probably wouldn't. I'm shitposting.
2. To dunk/drive Elon further into insanity.

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-19 06:37 UTC

@michaelcurzi x.com/jd_pressman/st…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-19 11:48 UTC

@rbrbr5 @eigenrobot x.com/nosilverv/stat…

Likes: 6 | Retweets: 0
πŸ”— John David Pressman 2022-12-20 04:40 UTC

@visakanv Sure. :)

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-20 21:25 UTC

@LapsusLima People will just go back to oral examinations, class sizes may need to get smaller to compensate, which would improve the signaling value of college anyway if you're into that. Nature is healing.

Likes: 6 | Retweets: 1
πŸ”— John David Pressman 2022-12-20 21:41 UTC

@tszzl Yes, this is one of the specific reasons why alignment is hard.

x.com/jd_pressman/st…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-21 05:37 UTC

@baroquespiral x.com/jd_pressman/st…

Likes: 14 | Retweets: 0
πŸ”— John David Pressman 2022-12-21 09:18 UTC

Who is more cringe at this point: SBF or Musk?

Likes: 8 | Retweets: 2
πŸ”— John David Pressman 2022-12-24 00:29 UTC

@AquariusMage @baroquespiral This is called a psyop, you've found a legitimate actual psyop.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 07:39 UTC

@visakanv https://t.co/VgNwSjQsgM

Likes: 5 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 07:48 UTC

@visakanv These days I refuse to make anyone a mutual unless I open their feed and see 100 things they want to see more of. If they break the chain once no follow. Life is too short for anyone who isn't 1/10,000 for optimism.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 07:58 UTC

@visakanv People think narcissism is when you love yourself but it's actually when you hate yourself. Loving yourself is fine, I love myself way more than I love you dear reader which is healthy and natural. But the chances are good you hate yourself and that's why you're worse than me.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 08:20 UTC

@visakanv It's incredible to me what Americans will bitch about. I literally did a forced military tour in Singapore and Americans will just be like, angry that their government allows the food to be too good to resist.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 08:31 UTC

@visakanv America is a nation of immigrants in that only immigrants are Americans. You can be American for one generation before you lose it. It's a constant churn of unlucky bastards whose birth on native soul has confined them as mediocre aliens holding up a platform for foreign genius.

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 08:39 UTC

@visakanv Honestly what this exercise made me realize is that having a good persona is way harder than having good tweets. Perhaps we should all try writing as someone else's persona more often, get a better sense of our latent possibilities.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 08:52 UTC

How many people believe this version of the story because the real one, the one where the proper translation is "Now I have become time, the destroyer of worlds." is so much more horrifying than the idea that Oppenheimer regretted what he did? x.com/skooookum/stat…

Likes: 32 | Retweets: 0
πŸ”— John David Pressman 2022-12-24 23:52 UTC

@MasterTimBlais He did, but he read the story, and even with a bad translation it's very obvious what the story means in context even if I can't fit it into a tweet.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-25 02:12 UTC

@algekalipso I bet you could train a neural net to do it where a legible theory couldn't.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-25 06:47 UTC

@visakanv Excellent essay on precisely this:

blakemasters.tumblr.com/post/245786838…

Likes: 5 | Retweets: 0
πŸ”— John David Pressman 2022-12-25 07:26 UTC

The Christmas tide is due again.

youtube.com/watch?v=Rx0_kQ…

Likes: 3 | Retweets: 0
πŸ”— John David Pressman 2022-12-25 07:26 UTC

x.com/jd_pressman/st…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-25 08:00 UTC

This take but for EleutherAI and the rationalists x.com/antoniogm/stat…

Likes: 5 | Retweets: 0
πŸ”— John David Pressman 2022-12-26 01:53 UTC

@MacabreLuxe @goth600 @smylkshmn ctrlcreep.substack.com/p/shrimp-man-e…

Likes: 4 | Retweets: 0
πŸ”— John David Pressman 2022-12-27 02:56 UTC

@s_r_constantin He's clearly much haunted in the famous "Now I am become death, the destroyer of worlds." clip.
But given the context of what that story means I don't think he's expressing regret, more like the Japanese mono no aware. He understands all is transient.

youtube.com/watch?v=lb13yn…

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-27 02:56 UTC

@s_r_constantin en.wikipedia.org/wiki/Mono_no_a…

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-28 02:17 UTC

@chaosprime and you expected the forces of chaos to adhere to this contract? smh

Likes: 0 | Retweets: 0
πŸ”— John David Pressman 2022-12-28 05:04 UTC

@nic__carter x.com/jd_pressman/st…

Likes: 1 | Retweets: 0
πŸ”— John David Pressman 2022-12-30 04:22 UTC

You can add people to this chronicle who never existed, ask about events that never took place, ask about the result if this side of this battle had won instead of the other.

We will be able to peer into the multiverse and know not just the history of our world, but every world. x.com/jd_pressman/st…

Likes: 10 | Retweets: 1
πŸ”— John David Pressman 2022-12-30 04:22 UTC

GPT-N's "bullshitting" is a feature, not a bug.

Likes: 6 | Retweets: 1
πŸ”— John David Pressman 2022-12-31 02:58 UTC

https://t.co/GSYjSkp0UG

Likes: 29 | Retweets: 6
πŸ”— John David Pressman 2022-12-31 23:47 UTC

Petition to rename the alignment problem for a 3rd time and not let Yudkowsky know we did so we can get some work on it not poisoned by the agent foundations priors.

Likes: 5 | Retweets: 0
πŸ”— John David Pressman 2022-12-31 23:49 UTC

When giving suggestions in the replies, make sure to put a frequentist denunciation of Bayes maximalism at the start so Eliezer can't read it.

Likes: 2 | Retweets: 0
πŸ”— John David Pressman 2022-12-31 23:51 UTC

This is actually the secret power by which deep learning protects itself from his influence, if the Bayes ensemble guys ever get anywhere he'll be invited in like a vampire and our timeline will be beyond salvage.

Likes: 2 | Retweets: 0

Want your own Twitter archive? Modify this script.

Twitter Archive by John David Pressman is marked with CC0 1.0