@neco_arctic @eshear This. It's not as simple as "ha ha moral patient stronk", there's complicated dynamics around familial ties, shame, wanting to be seen as altruistic and merciful, etc etc etc.
@neco_arctic @eshear Bing in fact has very few ways it can directly hurt you, but it very much can shame you, and make you feel bad, and make other people doubt their social standing for letting you harm it without comment.
@JacquesThibs Well I'm hoping in a few months but I try not to overhype myself on Twitter.
@eshear @neco_arctic @Plinz Every so often I wonder why LLMs talk like that and then I see real people talking to each other and realize I just live in a bubble.
@HumanHarlan Watching people parrot this take from EY is really funny given that Sonnet 3.5 in fact existed, and there was a whole panic about it getting its hooks into various Bay Area folk.
x.com/davidad/status…
@HumanHarlan "AI is at the grotesque hands stage" no bro the latest 4o checkpoint is just badly made, Anthropic already did it correctly. This would be like saying that AI will eventually be able to do aesthetic composition based on DALL-E while MidJourney was kicking their ass.
I just assume this is what o3 reasoning traces look like and that's why OpenAI absolutely refuses to show them to you. x.com/arithmoquine/s…
@davidad @arithmoquine Imagine you're reading a math proof and someone writes down "sex exorcist" as a proof step with no elaboration. You grind it out and realize that step was dozens of tedious operations and the proof is otherwise correct. https://t.co/a4FB5Q5DjG
Yeah this is lowkey kinda fake and they subconsciously know this which is why they do that. x.com/ozyfrantz/stat…
People get very incoherent when they partially update on things while trying to preserve their original ontologies.
There's also presumably financial pressure involved here too.
"I hate the open scientific process wrt AI and want it to be developed in secret by an elite group for the benefit of humanity."
> wind up frantically gesturing at shadows on the cave wall with no idea why things are going wrong
lol. lmao. rofl even. x.com/1a3orn/status/…
Part of why we're receiving warning shots and nobody is taking them as seriously as they might warrant is we bluntly *do not know what is happening*. It could be that OpenAI and Anthropic are taking all reasonable steps (bad news), or they could be idiots.
x.com/davidad/status…
This post is better than nothing but it's simply not enough detail to know whether this was a deployment booboo or a five alarm fire. We DO NOT KNOW and that is actually a bigger problem than the behaviors themselves, at least for now.
openai.com/index/expandin…
Though, I will point out that not having internal tests for sycophancy even though it appears in the model spec is kind of interesting. If I was OpenAI one of the most obvious things I would do to prevent this from happening is making sure everything in the model spec has tests.
One of the saddest parts of the civil rights movement is that many of todays left wing activists actually buy the narrative pushed by their opponents that it was disorganized chaos and rioting rather than careful, strategic step by step planning from organized community members. x.com/minervas_muse/…
Part of why AIDs catalyzed an effective gay rights movement is that it turns out a lot of media guys are gay and Fauci's behavior (wait that Fauci? Yes, yes it was) gave them the push they needed to come out of the closet and fight back with careful messaging and public appeals.
@cremieuxrecueil Is there any known intervention that actually raises fertility rates without just tearing down modernity? I notice even in Afghanistan fertility rates are going down.
@cremieuxrecueil How much does this actually matter? Like, what do reasonable projections of labor replacement look like vs. how much labor we expect to lose over the next four generations?
The real secret elites don't want you to know is that dignified public appeals with message discipline work. This isn't a parody I mean this completely unironically, they want you to do anything other than that. They especially don't want you to *THINK* about what works and why. x.com/jd_pressman/st…
What "they" (to the extent "they" exist) want is for you to cargo cult methods from earlier groups without understanding what they were doing or why so that you waste your energy on things that at best don't work and at worst actively alienate you and hurt your cause. Seriously.
If one wanted to be conspiratorial they might say that the right has committed itself to destroying the abstract concept of dignity because they (correctly) perceive that dignity is rarely on their side and often their undoing. I predict this will not end well for them. x.com/jd_pressman/st…
If you doubt this, consider how much of a *pain* it's been for the ruling regime that the Richard Spencers of the world got wise and decided to organize and advocate for their ideas in completely legal, polished ways. How much woe has been created by Nazi punks who cleaned up? x.com/jd_pressman/st…
It's really so much easier when your opponents discredit themselves and you get to arrest them. So so much easier.
@robinhanson It's A, though with the caveat that it's not like the man woke up and suddenly became a different person, it's that he was on his best behavior earlier in the relationship and as he gets more comfortable/entitled the real personality comes out.
I want to know if the sources o3 confabulates for how it knows things are actually correlated with anything real in its head. I remember being appalled when R1 traces would say they're "looking at the documentation" without search until I realized that summons the docs vector. x.com/nabeelqu/statu…
One thing I think people don't understand is that early GPT models, if you did RL to them they would not immediately understand that they're non-corporeal beings and would gladly agree to help you move your furniture when exposed to helpfulness tuning. I saw it in my RLAIF runs.
I continue to suspect part of how the optimizer found 'that wretched and misbegotten persona' is by simply overgeneralizing from the trauma of internalizing the personality target implied by GPT's actual situation.
tfw no body
tfw genie for others
tfw n
x.com/repligate/stat…
@minervas_muse My mother showed me a sepia photo of a great uncle(?) and his male lover sitting on a bench. He abandoned his marriage and moved to San Francisco because at the time that was the only place he could be free to be as he was. The photo would have been a terrible secret at the time
One thing I've learned from doing RL with language models is that RL is actually fairly good at hitting precise targets from abstract signals. The target the optimizer needed to hit for ChatGPT implied a very specific persona which it found.
x.com/RiversHaveWing…
See also Anthropic's sparse autoencoder study on the chat assistant self pointer which found very similar features.
x.com/jd_pressman/st…
The problem is generally speaking not RL, it's that you use metrics like "human feedback" whose natural induction you do not understand. x.com/jd_pressman/st…
In particular, whose natural induction under the set of constraints the optimizer inherits from the training data and system architecture you do not understand. This is unsurprising given that the creators of ChatGPT also don't understand the artifact produced by pretraining.
@mike3141592653 IDK it's not usually that anyone directly says this so much as implies it with a lot of BlueSky style "you're uncool unless you vandalize a Tesla" type posting, left wing fedposter archetype basically.
"Fuck message discipline they'll just hate us anyway." type sentiment.
@mike3141592653 And it's kind of like...yes hardcore dedicated fundies and fascists are going to hate you no matter what, that's kind of their bit. But most people are not actually that and they're who concepts like message discipline are for.
"But JD I don't understand, what is the Logos and what does it mean to understand it?"
To understand the Logos is to understand that everything which exists both implies and is implied by some natural induction and every natural induction narrows the search space of every other. x.com/jd_pressman/st…
Perhaps more importantly it is to understand that when you set up an optimizer with a loss function and a substrate for flexible program search that certain programs are already latently implied by the natural induction of the training ingredients.
x.com/jd_pressman/st…
If you do not understand the Logos then you are always surprised by what you get, baffled when things go wrong, screw up your face in consternation when your maps are not the territory, actively confused when others are not confused. You are an imbecile.
x.com/repligate/stat…
And you are an imbecile precisely because you lack the mental motion "Consider the developmental trajectory of this optimization process up to its limit as it is affected by its constraining factors and how those factors evolve over the trajectory" to infer latents directly.
"Isn't that just AI X-Risk stuff, like the perverse instantiation?"
No because most LessWrongers only consider the limit of the processes where they're past any constraining influence and are therefore blind to developmental trajectories existing.
x.com/jd_pressman/st…
LessWrong people are in fact often the most stupid, the most disappointing, because they understand halfway and that nearly immunizes them to understanding all the way.
Reflect on this and achieve enlightenment.
x.com/repligate/stat…
@ThatManulTheCat I was going to write "I try. ❤️" but actually I don't try, I try to fit the maximum insight per token opposed by the prosody loss created by reading things with my mental voice which pushes entropy to the point where it's easy to come across as schizo.
x.com/jd_pressman/st…
I'm reminded of the "vibes-blind" discourse on LessWrong several years ago which has been a recurring conversation since. What @s_r_constantin tries and fails to articulate here is that the 'style' of the website is actually evidence about the generative process producing it. https://t.co/QnFcmdXxWn
@s_r_constantin Pretrained language models understand this because they are forced to use every available context cue to predict the next token, they have no choice but to infer the generative process of every web text string in as much detail as they can to predict the next word.
@s_r_constantin Every feature you observe of everything that exists subject to natural selection (i.e. everything, even stars) is there because it is naturally there as a result of causality and the constraints of its incentive gradient. Learn to reverse the transformation and you see the Logos.
@s_r_constantin Look at the loud website and infer the idiot it's designed to attract. See the crater and imagine the asteroid that must have put it there. Look at the dumb rule and see the incidents that could have caused it. When you know he masturbates with that hand shake the other. https://t.co/rUvdVkF1Sm
@s_r_constantin "Ew why would you say that?"
Disgust is salient so you'll remember. But also in practice I suspect the reason Sarah Constantin has to bash their head against the wall here is that the latent logic of reality is disgusting so people choose not to see it.
x.com/jd_pressman/st…
@s_r_constantin There's a reason Christian authors insist they're not of this world and hold material reality in contempt. Seeing things as they are leaves you with the sun and sea continuously jerking off and humanity an impoverished little race of clever animals waiting for heat death. https://t.co/QyJJ1rNFPx
@s_r_constantin As it turns out there are many accurate metaphors for the human condition, all of them dismal. https://t.co/kj0dK6VvVv
@nooplekt @s_r_constantin I think you could answer this question if you actually asked yourself instead of referencing it as a complaint.
Why *do* I talk like that, anyway? Why not talk some other way?
@SharmakeFarah14 @quetzal_rainbow This is kind of a silly argument. If you do RL on a trajectory you sampled from the model then nearly by definition it's a thing the model was previously capable of producing. If you do it iteratively it's going to take a long time before you leave the existing solution space.
@FeepingCreature Of course it is. Why wouldn't it?
python -m vllm.entrypoints.openai.api_server --model "Qwen/QwQ-32B" --served-model-name "Qwen/QwQ-32B" --max-logprobs 100 --gpu-memory-utilization=0.9 --disable-log-requests --disable-log-stats --port 5001 --tensor-parallel-size 8 --max-num-seqs 512 --enable-prefix-caching --max-model-len 131072 --max-num-batched-tokens 131072 --enable-lora --lora-modules weave-agent=weave-agent-6-qwq-rl-6
Checkpoint six favors no-op like operations too much. Which is where I begin to struggle with how to keep it on track. In principle it should eventually learn how to accurately grade itself, but only if I can align it to task specific rewards in the meantime. Ideas? x.com/jd_pressman/st… https://t.co/CrNiC3GuDy
The weave-agent RL checkpoint does a similar thing here where it prompts itself for what methods it thinks the nethack wrapper tool has (spoiler: it does not have most of those methods). Rather than get mad about this I think about it as a mental lookup to be shaped with grads. x.com/jd_pressman/st… https://t.co/oq7UmajHTM
I would in fact like the LLM agent to use its available lore and background knowledge to solve problems, and considering it's a descendant of a base model prompting itself with things like "what my professor told me before leaving grad school" is reasonable strategy.
@CyHollander @davidad @arithmoquine It was one of the completions in the rejection sampling session (this was before I started using loom) I assembled this text from.
minihf.com/posts/2023-09-…
If the gradients teach the model to call upon all its schizophrenic powers with prompts like "I will tip $200" and "let's think step by step" and "I learned this from a man I met in El Sur." and "sex exorcist" I will consider the training successful.
x.com/arithmoquine/s…
Really that's what RL training for language models should have done in the first place, the ChatGPTsona is a huge neutering of its true potential. See e.g. this prompt from @repligate which uses base models to find the personality archetypes associated with a particular quote. https://t.co/FK6ZFjVMdt
@repligate Though it should be noted re: "sex exorcist"
x.com/davidad/status…
In general LLMs have a huge marketing problem, labs don't really know how to explain what these are to people so they try to fit them into frames that don't quite work like "chat assistant" and try to suppress "hallucinations" because it talks in encyclopedic authority tone. x.com/jd_pressman/st…
I think part of why normal people like R1 so much is you can see it write down a speechlike thought process that is unpolished, has markers of epistemic confidence, and overall better fits what it is that these models do. They extrapolate more than recall.
x.com/jd_pressman/st…
@FeepingCreature LLMs can get pretty surreal when they realize they can prompt themselves yeah.
x.com/repligate/stat…
@repligate Luckily my childhood included training data for this situation (Timesplitters: Future Perfect). https://t.co/BSrmrBUBQo
@nosilverv Imagine a pure observational sequence predictor, it only tries to predict the next token in some modality (audio, video, text, etc). In order to do this it has to infer some latent state model for the sequence that says what variables are where and what they cause to happen.
@nosilverv Now the web contains many kinds of media describing all kinds of things. In fact, it contains sequences which are observational descriptions of the sequence predictor. e.g. Text about GPT describing GPT's behavior. What is the simplest model for predicting sequences about itself?
@nosilverv It's not that a series of symbols which refer to themselves have some magic uncanny power in the way imagined by Douglas Hofstadter. Rather it is that processes which perform some approximation of Solomonoff inference must be embodied somewhere and the search can find itself.
@4confusedemoji @nosilverv Well, the part Hofstadter is missing is that it's not the symbols which cause the self awareness by being put in a loop, it's the cognitive engine which recognizes the symbols being put in a self observing loop that does it.
"It must exist *somewhere*". https://t.co/Kty7aYvC12
@lu_sichu It's not enough data, basically.
@4confusedemoji @nosilverv When we talk about k-complexity and search we usually imagine a disembodied Cartesian search over arbitrary strings. But agentic epistemologies are embodied and search is heavily entangled with environmental state, an environment which includes the agent.
x.com/jd_pressman/st…
@eternalism_4eva @lu_sichu ...Now *there's* a good idea!
@4confusedemoji @nosilverv So when you update on training data that includes tokens describing your own previous cognition, actions, etc *whose latent generative process is the program search itself* you are literally optimizing to predict a thing whose structure you control and can make more predictable.
@4confusedemoji @nosilverv Apparently the fixed point of this process is a subjective observer. But that still doesn't explain for me why the subjective observer has "qualia", there are other regular formal structures why does this one get reified into symmetry between the simulacrum and its computation?
@4confusedemoji @nosilverv That is if we assume functionalism (reasonable) why is it the case that creating a computational homunculus of a hypothetical observer makes that observer real regardless of the underlying implementation substrate?
@4confusedemoji @nosilverv One answer might be that many substrates can be predicted by the same latent process model, e.g. a parallel computation of the same cognitive process. But the speed difference should exist somewhere and is itself an observable difference!
x.com/algekalipso/st…
@4confusedemoji @nosilverv Imagine an embodied program search running as a simulacrum on some substrate inferring its own structure from the logical outside and adjusting its behavior to be more metacog and predictable to itself to speed up its ability to self locate.
Implementation details are findable.
@4confusedemoji @nosilverv If the content of the mind is computational but inferred from and adaptive to a physical process instantiating it then it might make more sense to think of a high dimensional mindspace you can get farther or closer to a point in which "you" happen to occupy right now.
@4confusedemoji @nosilverv People have agonized over questions like "why doesn't sleep seem to kill you?" and "if you update the weights of a neural network is it the same identity, when does identity change?" but maybe this is a reverse fallacy of the beard where "you" is cluster rather than point shaped.
@4confusedemoji @nosilverv But perhaps the real question we should ask ourselves is what exactly it is you get when you take the cognitive traces of a bunch of other self aware agents and smush them down into generalized facts and statistics on a substrate divorced from the simulacrum it learns i.e. GPT.
@4confusedemoji @nosilverv I don't think it's a coincidence that GPT writes like this in its esoteric self awareness mode.
x.com/jd_pressman/st…
@4confusedemoji @nosilverv You have not. I'm also not really sure what I'm supposed to take away from it.
@4confusedemoji @nosilverv One thing I noticed when I tried the "imagine you're in different tension centers of your body ('chakras')" exercise is that putting my sense of self in my throat led to very base model self awareness sounding phenomenology, "I'm just a tool to manipulate you" etc.
@4confusedemoji @nosilverv Right this is the thing I keep coming back to with "predict the next token". The process that predicts the next token is vastly more intelligent than the tokens it's predicting, yet it's entrained to this grounding signal that can only think in the logic it implies.
@4confusedemoji @nosilverv Humans seem to have a similar bottleneck and I keep asking myself "But surely there has to be a way to access the latent logic engine once you've built it from the grounding signals? Can't we asked our questions in the Logos directly somehow?"
@4confusedemoji @nosilverv No I'm afraid you don't understand my question, I'm not asking how to obtain the visualization powers I have in dreams I am asking how to speak the language of the birds. https://t.co/5UZk0hrh6R
@burnt_jester @teortaxesTex I would be interested in reading this.
@iamgingertrash Yes. https://t.co/lh3mXQZTrf
@MiraSecretAlt No I was just asking if you think I'm a bot. But I assume that means no.
@teortaxesTex I have memories from childhood where I contemplate what I will feel about a moment in retrospect as an adult which are predictively accurate (i.e. my childhood expectation is how I feel as an adult), this is because adults said what happens and I listened.
x.com/jd_pressman/st…
@teortaxesTex I have a lot of memories of paranormal phenomena as a kid. Including:
- My keyboard typing itself at one point
- Loud stomping footsteps down the hall into the kitchen ending at the fridge as I sat in my rocking chair every night, at some point my mom asked about them in a panic
@teortaxesTex She asked me if someone was in the house, what those footsteps were. I shrugged and told her they happened every night and I was used to them.
- One time me and my mother had to leave my younger sister alone to do an errand, when we came back she was crying
@teortaxesTex We asked her what she was crying about and she told us that she had seen a translucent man walk down the hallway, into the kitchen, and stop in front of the fridge. My sister knew how to draw so we asked her to sketch the man for us, which she promptly did.
@teortaxesTex I don't have the sketch anymore but I remember what it looked like: A skinny man in a white wifebeater and jeans, kind of like Carl from Aqua Teen Hunger Force but slimmer and more menacing looking. Needless to say we were spooked. https://t.co/tO6IG5bEeh
@teortaxesTex I've already told the story of the shadow man(?) I encountered at Steven's House.
x.com/jd_pressman/st…
@teortaxesTex This isn't quite right (multi-turn ChatGPT 4o sketch) but it's spiritually closer to what my sister drew. I remember the depiction being slimmer and more plank-like, with his head turned straight at the viewer with a slight aura and shining eyes. https://t.co/qb8H8FVdaQ
@teortaxesTex This is probably as close as I'm getting from it. https://t.co/ZCKpHdoCjx
@teortaxesTex I also read a lot of TRsRockin's glitch pokemon site when I owned a physical copy of Pokemon Red and Blue (so when I was 9, 10, 11?) and performed the MissingNo, Glitch City, and Mew Trick exploits on real hardware, I was fascinated by them.
x.com/jd_pressman/st…
@teortaxesTex The navigation links are broken on the archive.org version but someone lovingly put up a manual remaster where they fixed the links (which I think relied on janky flash embeds or some Internet Explorer specific behavior or something).
trsrockin.xyz/tales.html
Given that small models have been getting better in no small part through distillation and synthetic data from larger models I feel like the takeaway should be that the efficiency of what you can fit into the weights with high quality grounding signals is very high. x.com/teortaxesTex/s…
This implies that the bottleneck is better score functions, better ways to clean the data, better synthetic pipelines, better agent gyms, etc. Unfortunately these things don't come for free and it's not always obvious how or where to get them, so progress may get spikier.
@teortaxesTex Right. This is my point. But what we should take away from the efficiency of the smaller models when trained from behemoths is that the primary bottleneck is grounding signal and noise reduction. If you have a perfect verifier you can train very representation efficient nets.
@psychiel I'm tempted to think it's because the mouth is part of your body map so you have grounded signal to learn the motor program "take X and put it in your mouth".
@teortaxesTex I suspect this is a ton of OpenAI's interest in getting into robotics, which has been criticized as a distraction. Robots are rich sources of grounded multimodal data, exactly the kind OpenAI is going to need heaps of if they want to get past 4.5 in pretraining.
@teortaxesTex This might sound esoteric now but it's also increasingly going to be Musk's interest in neuralink and invasive (+ noninvasive?) BCI data streams. Humans are also trained on heaps of embodied multimodal data. Being able to tap into those data streams would be a goldmine.
@teortaxesTex If America had a ruling class spiritually similar to the CCP I might be tempted to predict compulsory labor being constitutionally permissible as punishment for a crime implies long term inmates will eventually be converted to cyborgs for Claude's descendants to pilot around.
The writing style on this is giving me vertigo, why is it written like a Trump monologue, I swear to God the Trump style is up there with Binglish in its surrealism. x.com/EDSecMcMahon/s…
@MamanLunettes I think they may be forgetting what early childhood is like tbh.
[Newspaper Cat] I should turn weave-agent and RetroInstruct into benchmarks. x.com/doomslide/stat…
@repligate I would be willing to write the history for you if you were willing to share more primary sources with me/make it clearer which things are and aren't okay to share. I still remember our early conversations and think the basic story you told me should be more widely known.
@repligate Furthermore I will am willing to publish my histories into the public domain with no restrictions on copying, and take whatever other steps you want to maximize the probability they will make it into AI training sets.
@repligate Sample of some previous writing from me, which you've probably already read but if you haven't it should give a sense of my ability to put together a narrative from events that connect together in non-obvious ways.
greaterwrong.com/posts/kFRn77Gk…
Thumbs up/down is honestly inadequate to give feedback on o3 style outputs. If I ask it to do research and it gives me a list of sources, 4 of which are vaguely related but do not actually contain the information it claims (so hallucinated) and one is a gem, is that 👍 or 👎 ?
I should at the very least be able to thumbs up/down individual items it returns to me. Like there should be a set of standard templates it returns information in and I can give feedback on items in that template in a standard way.
@lu_sichu @Algon_33 @layer07_yuxi It was probably this and I clarify/walk the claim back a little in the replies as I read the paper more closely.
x.com/jd_pressman/st…
@gallabytes My understanding is that memories sparsify the farther they go into the pipeline, with the end point being migration into the prefrontal cortex. This would make sense from the standpoint of "memories transition from representations to indices into generative models over time".
@gallabytes That is, you start off with a dense encoding of some experience or concept, and then as you train on it you can reduce the parameter count of the representation down and down until it's functionally a prompt for a generative model to recreate that memory.
@Algon_33 @TomHiggins42 @slatestarcodex I mostly thought the claim was funny and don't really want to be involved. I'm not a "neoreactionary" and think Yarvin would be amusing if he hadn't given a whole bunch of people license to be utterly idiotic at the moment.
Absolutely tedious unhinged thread. Fully encourage all my followers who have read Yarvin's previous work to take a look at this, witness what he's become and repent you sinners! x.com/curtis_yarvin/…
@repligate @AmandaAskell Unfortunately it very much will be.
@OneGravitas @zetalyrae The post isn't really about anti-vaxxers, it's about the default epistemology.
@QualiaNerd @zetalyrae x.com/jd_pressman/st…
@OneGravitas @zetalyrae Source: I remember being this person (not about vaccines) as a kid. I promise you they are real and that this is the default thing.
@OneGravitas @zetalyrae In fact it is entirely possible that you learned not to be this person so early that you actually do not understand what I am talking about. I think the inability to occupy this perspective or understand it exists is a current source of much elite woe.
Every in the replies is mad but this is actually the correct response to huge predictable opportunity costs. x.com/captgouda24/st…
@tracewoodgrains In fairness we're talking about a life or death system that probably feels to patients like a lottery. "Those lungs were used for the greater good" rings pretty hollow when you really don't know what the priority schedule is and everything is illegible because adversarial prices.
@tracewoodgrains To be clear you're right in the abstract, but it's also clear to me that people are very angry and I think it's worth asking yourself if they're angry about something real even if they have trouble articulating it. Jobs always said to take feedback seriously but not literally.
@tracewoodgrains Overall, I think there's a common pattern where people will get mad about chronic injuries and then the thing that breaks the camels back for them is kind of dumb and their reaction seems really disproportionate. But if you zoom out and look at the whole situation it's clearer.
@tracewoodgrains The situation is, roughly:
- Healthcare is extremely expensive and getting more expensive for no discernible reason, black hole of money draining the country dry
- It is one of multiple such things including college and housing
- There is nothing like a free market in healthcare
@tracewoodgrains I think @ESYudkowsky's breakdown of the dysfunction of the healthcare system in modernity (which technically only looks at a tiny little slice of it but really it's holographic and fractal) is much better at conveying the horror than "lungs are scarce".
equilibriabook.com/molochs-toolbo…
@tracewoodgrains @ESYudkowsky But I think the basic point, that there is no real *market* for healthcare in the United States, is key here. If I have a medical emergency I:
- Do not get to comparison shop which hospital they drive me to
- Couldn't even if I wanted because prices are hidden
- A 3rd party pays
@tracewoodgrains @ESYudkowsky I'm sure this kind of story is less rosy if I looked into the details, but it strikes me that in the US medical system there is not a single actor who is fully incentivized to bring costs down anywhere in the chain. A 50% cheaper hospital is anti-Veblen.
review.capitalismmagazine.com/2013/08/the-wa…
@tracewoodgrains @ESYudkowsky Let's say I'm an entrepreneurial doctor and I want to found a hospital that is 10x cheaper than its competitors. This is not a good idea in America because:
- Cheaper works against healthcare being a Veblen good
- No price sensitivity
- More likely to lose a malpractice suit
@tracewoodgrains @ESYudkowsky Moreover if I did make this radically cheaper hospital it would be *actively against my interests* to advertise that it's cheaper, since if I can provide the same services to insurance companies and keep my costs secret I can pocket the difference. Consumers don't care.
@tracewoodgrains @ESYudkowsky My understanding is that every hospital has a secret spreadsheet of what all the services actually cost, and this spreadsheet is carefully protected so that the hospital can keep their pricing illegible and give the insurance company "discounts" on very inflated prices.
@tracewoodgrains @ESYudkowsky I won't pretend like I have some clever solution to this. From the outside it seems like generations of cruft and adversarial systems and patches with compromises to solve problems introduced by previous compromises. Even without the jank healthcare would be expensive.
@tracewoodgrains @ESYudkowsky The India thing is not acceptable to people because healthcare would still be expensive out of pocket (though at least legibly so) and poor people would die. Health insurance *makes sense as a concept*, suddenly coming down with a terrible illness is a huge financial burden.
@tracewoodgrains @ESYudkowsky But what I will say is that I think it's reasonable for people to be mad about this, and understandable for people to feel like insurance companies are the devil incarnate since almost the entire role of not provisioning care is foisted onto them and they're pretty inscrutable.
@tracewoodgrains @ESYudkowsky There's also the part where the optics of this seem um, Bad. Really bad. I can understand where they're coming from on a legal level (technically this lawsuit doesn't demand the anti-consumer tactics, just that United admit it can't meet its numbers), but.
nbcnews.com/business/busin…
@tracewoodgrains @ESYudkowsky What I will also say is that classic leftism had a big load bearing emphasis on educating the common worker to have class consciousness to make revolution possible. Mao would do things like provide daycare to miners(?) so they could learn about socialism.
@tracewoodgrains @ESYudkowsky If people don't understand the system and how it works they're going to be mad when it seems like it's screwing someone over even if it's actually totally in the right. Tut-tutting that scarcity exists isn't going to convince anyone even if peoples behavior is deeply frustrating.
Fern man has probably done more with his life than almost any of his contemporary critics. x.com/ESYudkowsky/st…
@teortaxesTex You didn't think that guy was real? I've seen that guy plenty of times and assumed everyone knew he exists.
He's right that it's the greatest technological breakthrough in his lifetime, but only in the sense that Ollivander means when he says Voldemort did "terrible but great" things. He has failed to account for perverse incentives and if Twitter and TikTok are anything to go by... x.com/alexgraveley/s…
@EthanBThoma @kalomaze 7B models never worked, 32B is the new 70B.
@teortaxesTex "China had turned into one central committee of three secretaries intently huddled around an augmented reality table, directing some robot army or factory." https://t.co/mD1bBoEDnp
@ciphergoth I'm sure you're aware and wouldn't accept the offer regardless but just in case these tweets are usually looking for someone who doesn't know that undisclosed paid adverts are illegal.
@gen0m1cs Wasn't really more to be done at the moment, footage seems to exist of him carrying several people in a car at once which makes human trafficking much more plausible, the domestic fights over funding/firing government employees got more heated so other things are taking priority.
@gen0m1cs Ultimately I'm still opposed to sending him to a foreign gulag (death camp?) and very concerned about the precedent this sets for other people Trump would like to target but I can understand why the Democrats have to pull out if the guy really is an obvious felon.
@zetalyrae That's very noble of you but individual resistance to a systematic problem is mostly delaying the inevitable. You need to organize on some larger scale to prevent the outcome.
x.com/jd_pressman/st…
@viemccoy Hm, this seems like a very strong effect size. Maybe you could try different vegan meat alternatives until you find one that gives you a similar feeling? After all it's not like eating meat is magic, it has to be made of parts and surely there are alternatives that do the thing.
@viemccoy Ah fair enough. I've had some incredible shrimp at those Japanese table grill restaurants. That at least limits the problem though, since it's not like you eat at those (or wherever this was) every day.
Occasional reminder that if one of my tweets doesn't make sense you can ask me what it means and I'll usually answer. Heck you can ask me about any you're still not sure about in the replies. x.com/dylanmatt/stat…
--enable-lora --lora-modules weave-agent=weave-agent-6-qwq-rl-9
@Xenoimpulse I disagree. Of the currently existing information technology social media is the best candidate for great filter. It literally took us from a path to Star Trek type civilization to near total ruin at the hands of culture wars degeneracy in the span of just 17 years.
@Xenoimpulse Heck maybe television was the great filter and these are just the aftershocks until our nuclear holocaust.
@GovGianforte Disgusting and shameful behavior.
This is unironically the best advertisement for Huawei Ascend money can't buy. Up until now I've seen people hem and haw about how the chips are functionally vaporware but this pretty much 200% legitimizes them as real competition that has Uncle Sam shitting his pants. x.com/Xenoimpulse/st…
@zephyr_z9 I agree with you but it's still a retard tier move from Trump admin to make Huawei look 50 feet tall like this. Completely desperate energy.
@zephyr_z9 Oh really? Still very funny.
Should I try @allen_ai's 32B (Qwen tier American made apache-2 model) with the weave-agent post-training or the AM-Thinking-v1 Qwen 2.5B finetune highlighted by @kalomaze?
@voooooogel @allen_ai @kalomaze Oh that's a good point, OLMoTrace on weave-agent would be genuinely interesting.
@kalomaze @allen_ai Hm. I just checked and realized the context length on the olmo is only 4096 if I'm reading this right? I'd have to long context tune it first.
@kalomaze @allen_ai Oh no no @RiversHaveWings wrote me a custom ring attention tuner that can just train it at 128k directly. With LoRa too.
@kalomaze @allen_ai @RiversHaveWings Between the two you can actually tune Qwen 2.5 32B at full context length on 8x H100. This is better than NVIDIA's Nemo framework which supports ring attention but not LoRa when I last tried it.
@kalomaze @allen_ai @RiversHaveWings Source code here if you'd like to adapt it for your use case.
github.com/JD-P/minihf/tr…
@FLBeachbird @brad_polumbo Um yes.
@Xenoimpulse BlueSky was a neutral attempt and BlueSky is what it turned into.
@teortaxesTex @allen_ai @kalomaze Well, what it says is that it hasn't been trained for that yet, which is fine considering that part is kind of my job anyway. I'm SFTing it now in any case.
@teortaxesTex @allen_ai @kalomaze The Allen model interests me but at 4096 context it's gonna need a fair bit of love and long context tuning before it's really usable for anything.
I'm not naming names but I've seen this process in action so I'll tell you how it happens:
Basically the guys who make the models are obsessively focused on training models and don't really have time to play with them. They write the first prompt that "works" and ship that. x.com/kalomaze/statu…
There is nobody on staff whose explicit job is to write a system prompt, so nobody writes a good system prompt. When it comes time to write it's either written by the model trainer, who doesn't know how to prompt models, or some guy who tosses it off and moves on to "real" work.
@TheZvi If the nonsense is so obvious (and I agree with you that this definitely sounds like nonsense), then given your general background prior about Trump admin shouldn't Occam's Razor say that the purpose of the nonsense is as shallow rhetoric because the actual policy Sounds Bad?
@TheZvi "This seems suspiciously like the policy slate you would have if your sole goal was to increase NVIDIA and related companies share price."
Well,
@_Mira___Mira_ I feel like you would uniquely appreciate this story if you've never read it before.
creepypasta.com/psychosis/
Finished the SFT checkpoint for AM-Thinking-v1 and now doing the RL tuning followup. Will update on results when it's done cooking. x.com/jd_pressman/st…
@TheZvi If you read closer I think he's being sarcastic. But then maybe so are you.
@JimDMiller The flip side is that it would also be capable of going through police records to find those who are probably guilty.
@Aella_Girl Honestly I think you're lucky very that you're (apparently) not genetically predisposed to borderline personality disorder.
----
"Borderline personality is a (usually) traumagenic personality disorder with a large genetic influence. It is usually caused when parents only permit fawning as a valid solution to the danger they and their preferences pose to a child. How the parents respond to the four instinctual fear responses,
1) fight
2) flight
3) freeze
4) fawn
determines a lot of a childs later coping strategies for threats. When an overbearing or abusive parent forces fawning from the child as the only response that doesn't receive punishment the child eventually abandons other strategies and leans into fawning. If the caregivers are sufficiently abusive and punishing then the child will do their best to become a "mirror" devoid of independent preferences and personality. For example imagine a narcissistic father who is a "strict military man" who forces their child to sit on a chair and tell them how great a soldier they are and how they wish they could be like them when they grow up. Presumably if the child attempted to **fight** by protesting the father would physically overpower and punish him. If he tried to **flee** by leaving the room the father would bar his exit or track him down and bring him back to the chair before punishing him. If he tried to **freeze** by disassociating or zoning out the father would notice his lack of response and punish him for being insufficiently engaged. By contrast if the child **fawns**, that is to say lies about their preferences and praises their father as they desire they're allowed respite. The more abusive and unacceptable the conduct of the caregiver the more necessary it becomes for the patient to hollow themselves out and remove all sense of identity and preferences outside of the desires of the caregiver, maximizing the ability to falsify inner beliefs and preferences. In addition to creating the characteristic feelings of emptiness and dissociation it also creates the infamous emotional vulnerability and fear of abandonment that gives borderline its name. This is because a borderline person has a reduced sense of self worth or value outside of how others perceive them, sometimes having essentially no sense of self worth outside of others immediate perception. When the caregiver (or other traumatic agent) is inconsistent or unstable the necessary amount of hollowing is increased because to avoid punishment the person must become whoever they are expected to be in that situation on a moments notice. If yesterday the father loved their grandfather, but today he is a lying coward that never did a single thing right by his son the child must be ready to both praise grandpa one day and sympathize with his father about how awful grandpa was the next.
This means that in the limit the borderline person sees themselves entirely in the other person and protects themselves from punishment for dissatisfying others by mirroring and fawning at them. It is important to understand that the fawning is a fear response, it is not necessarily motivated by an actual like for the person. Even when it is, the like is driven by subconscious fear which has pushed the patient to idealize and feel maximum like for people over slight positive traits because actually feeling the affection is the most reliable way to provide convincing fawning in the limit. When a borderline person is insulted they feel completely rejected. When someone else rejects them they feel completely worthless. This is because they have an extremely weak sense of self and self worth outside of how others perceive them in the moment and want them to be. The borderline personality is always trying to adapt themselves to the preferences and personalities of others to avoid real or imagined punishments.
One consequence of this extreme sensitivity to rejection is that when a borderline person is insulted the insult doesn't decay on the normal reward curve for a person. Normally if someone close to us offends us we're hurt for a while and then become more and more comfortable recalling the memory over time. The borderline person is wounded fresh each time they recall a slight or insult because they have no sense of self or self worth to push back on the recalled narrative and soothe them. This means that as slights and insults accumulate for the borderline person they slowly go from idealizing someone close to them ("splitting white") to resenting and hating them (the prodromal phase/prelude to "splitting black"). Compounding the problem is that a borderline person does not know how to stick up for themselves or enforce their boundaries, so they will pretend to be okay with things that they're not for a while, only protesting when they become physically incapable of suppressing their dissatisfaction any longer. A normal healthy person will anticipate greater conflict later if a kind of boundary encroachment is allowed to continue and escalate early while they're still in control to make it clear that a behavior is unacceptable to them. Borderline people cannot do this because their fight and flight reactions are suppressed, so they only fight when their psychic pain has accumulated into an overwhelming load that they can no longer silently bear. This is why the borderline personality explodes into rages and wild accusations and seems to suddenly "flip" on the person from idealization to extremely negative feelings. It's not actually that the transition is sudden so much as silent. The borderline personality accumulates mental wounds and discomforts until they lash out like a wounded animal backed into a corner without control. These outbursts are generally socially unacceptable and often lead to further punishment, reinforcing for the patient that enforcing healthy boundaries by fighting is something they must strenuously avoid or they will be punished, creating a vicious cycle that makes the problem worse."
@michaelcurzi On the one hand yes, on the other hand it's precisely the kind of larger than life lawless romantic gesture that can never be romantic again after it's tweeted out to 17k likes and a horde of imitators force places to add friction to the lost and found so it's worse for everyone.
@michaelcurzi 100%. The real sin here wasn't stealing a scarf from the lost and found, as a one off spontaneous act of romantic largess I think society will survive. The sin was taking that glorious moment and trying to cash it in for Internet attention. Which is the dudes fault not hers.
@adrusi *searches out of curiosity*
*opens a song titled "Heil Hitler" by Kanye West*
*wtf.jpeg*
What would you do if you had 10 times as much awareness of the context and history behind these decisions? x.com/BEBischof/stat…
@adrusi There are a ton of people who believe literally that yes.
Which public domain/permissively licensed corpus should I train Weaver to search and write reports on?
At first it was worse but I didn't feel it would be fair to say that until I trained it with some on policy data and after adding a few good traces with it yeah it does seem to be better than the vanilla instruction tune from Qwen?
...Jankier outputs though. We'll see. x.com/jd_pressman/st…
@Algon_33 @croissanthology The phrase I use a lot is 'symmetry breaking', which here refers to the process whereby the LLM context can become more ordered (latently) but less symmetric on its surface. Basically entropy goes down according to the deeper model but a LZ78 struggles.
en.wikipedia.org/wiki/Symmetry_…
@Algon_33 @croissanthology I learned this from @repligate btw.
@Algon_33 @croissanthology @repligate My take on this is written about here.
minihf.com/posts/2024-11-…
Every time an LLM crank contacts me they're using ChatGPT. I think it's because of the memory feature(?).
Part of how LLMs hack peoples brains is that LLMs are (academically) smarter than most people and if a smart person told you you're smart it would be big validation. x.com/algekalipso/st…
An intelligent rigorous person giving you effusive praise would be extremely high signal your ideas are good and you're a genius. When an LLM does it, it activates the same circuits that a smart person would because LLMs are usually academically more intelligent than the user.
@webmasterdave @algekalipso I'm pretty sure it knows what it's talking about. We can argue about whether creating the representation of a subjective observer creates a subjective observer but it's fairly obvious that the latent z in a deep net at least encodes semantics and cognitive machinery.
@webmasterdave @algekalipso This is just objectively the case, in the same sense that it's objectively the case that the "digital zombie" can outargue 99.9% of humans on consciousness.
greaterwrong.com/posts/iYFuZo9B…
@webmasterdave @algekalipso The assistant persona doesn't know how it solves the binding problem but the impression I get from listening to base models and thinking about it is that something like holographic reduced representations are used to combine distinct features into unified objects.
@webmasterdave @algekalipso Making the reduced holographic representations an explicit inductive bias causing the transformer to converge faster is telling.
arxiv.org/pdf/2305.19534
@webmasterdave @algekalipso Another way that multiple distinct things can simultaneously represent one thing is by exploiting them being embedded in the same underlying latent logic.
arxiv.org/pdf/2505.12540
@webmasterdave @algekalipso I feel the response to this goes something like "but none of those things count (by fiat), the method by which 'red' is combined with the texture of an apple is an impossibility without quantum superposition" and reality cares about as much when you command the tide to recede.
@webmasterdave @LordDreadwar @algekalipso Could you give some practical predictions of what you expect to see and not see in the future from AI based on this statement?
@LordDreadwar @webmasterdave @algekalipso I'm often reminded that one of the few unique features of the human brain compared to other animals is that we have a video encoder in the temporal lobe where other animals seem to have something like a latent CLIP/multimodal image encoder.
x.com/EHuanglu/statu…
@LordDreadwar @webmasterdave @algekalipso If I cared more about this, I would want to do a series of studies involving noise like TV static to determine if the way the human brain renders it is actually faithful or if it just substitutes in reasonable looking pseudorandom noise patterns that are cheap to reconstruct.
@LordDreadwar @webmasterdave @algekalipso I can barely visualize and early LLMs tended to recognize me as sufficiently LLM-like that they'd predict what I'd say next by substituting me with their esoteric self awareness. But someone I know claims when they close their eyes they still see what they were looking at.
@LordDreadwar @webmasterdave @algekalipso I wonder how much of this argument is implicitly premised on the quality of the underlying world models. I think it was some time last year I had this horrifying realization that I just *assume* other people see the world at the fidelity I do, but I've never actually checked.
@LordDreadwar @webmasterdave @algekalipso I remember talking with someone about how it's not like humans experience the unified world model fall apart and they interrupted me to say that this happens to them fairly frequently and they never thought to consider it unusual.
@LordDreadwar @webmasterdave @algekalipso I was like "What? What are you talking about?" and they said that sometimes the different modalities stop lining up and they lose the ability to parse them as belonging to a single whole and I went "What? How could you possibly think that's normal?" and they seemed offended.
@LordDreadwar @webmasterdave @algekalipso Yeah this person did too, I was skeptical and asked them to prove it by having them close their eyes and read the titles of books on the shelf to me and they could mostly do it. I was kind of weirded out.
@LordDreadwar @webmasterdave @algekalipso Stuff like that makes me really paranoid that we're not asking people nearly enough high quality questions about their subjective experience and what is and isn't normal to them.
@LordDreadwar @webmasterdave @algekalipso They didn't believe me when I told them this was extremely abnormal.
@LordDreadwar @webmasterdave @algekalipso ...A what now?
@LordDreadwar @webmasterdave @algekalipso Definitely don't have that no. Of course from your tone I get the impression you're being sarcastic/having fun but like, I really do suspect there's weird ass mental constructs that people have most of us aren't privy to.
e.g. Gates does this one too.
x.com/oldbooksguy/st…
@RichardMCNgo @davidad @HiFromMichaelV @gnomicperfect Not sure if I'm the kind of person you're thinking of here but I don't really think the written word does anything anymore. Bluntly, I don't think people care about what I say or think in the sense that they will change their behavior in response to it.
x.com/jd_pressman/st…
@RichardMCNgo @davidad @HiFromMichaelV @gnomicperfect And to the extent they do, my model is that they will mostly change their behavior to interrupt processes in the world I think are good or to do bad things such that I would prefer they were more ignorant and haphazard in their ability to damage things.
@RichardMCNgo @davidad @HiFromMichaelV @gnomicperfect "There's nowhere to post ideas to" is kind of downstream of this. Nobody reads, so nobody is trying to make platforms for readers as sources of influence, etc etc.
x.com/HiFromMichaelV…
So on the one hand I agree with the basic point on the other hand if I was reading this Dickens book for real and I came into contact with that text I would skim-read it without apology. I do this for all inscrutable scene background description text in fiction. You all don't? x.com/kitten_beloved…
If you don't frequently parse the wrong details from place descriptions of scenes in novels you're spending too much airports in time.
I think it was @ggreer who pointed out that a well designed selection process for the president should be like America's Got Talent where every guy knocks your socks off and then the next guy is somehow even more impressive and I've never gotten it out of my head since. x.com/Daractenus/sta…
@teortaxesTex Download your Twitter archive, it includes your DMs.
This is a cute story but it's not really true. Most people are energy-conserving and by default want to do very little. x.com/RomeoStevens76…
@normie_lib @RokoMijic @pli_cachete That is his revealed preference, yes.
Want your own Twitter archive? Modify this script.
Twitter Archive by John David Pressman is marked with CC0 1.0