John David Pressman's Tweets - July 2023

@kandouss What would convince you?

@kandouss I don't want this to come off as a dunk because I admire the honesty, but isn't it sort of a problem if you have this extremely important priority that's made of beliefs/words that don't pay rent in anticipated experiences/predictions? One to be fixed immediately?

@kandouss But that's not what I asked. I asked what would *convince* you. What demo or kind of understanding are you looking for? For example, why is RLHF inadequate? What hypothetical version of it would be more convincing to you?

@kandouss Jailbreakability is almost certainly tied to overall capability (e.g. hard to imagine a 6 billion parameter model that can't be tricked besides just nixing certain parts of latent space), but if you had a method that fixed confabulation and jailbreaking what would still be wrong?

@kandouss My biggest outstanding problem is inner alignment/detecting mesaoptimizers/things in that cluster, yeah. I kind of have an ugh field around going into detail because it feels like I'm arguing with peoples bad priors more than anything else.

x.com/jd_pressman/st…

@kandouss Lets go through List of Lethalities. My rebuttal to point 1 is something like "AlphaGo relies on being able to (mostly) compute the reward for the domain it's learning, text isn't like that, can only eliminate half the hypothesis if bits are uncorrelated"

x.com/jd_pressman/st… https://t.co/8rPRoiLL2b

@kandouss This part seems fair enough to me and I don't understand why people usually focus their rebuttals here, it seems like one of the most "well duh" parts of the whole discussion. https://t.co/s0rm7Y42yB

@kandouss This one is kind of a tautology? If you accept the premise that there will be a singleton (I'm unsure) then obviously the creation of the singleton has to be done correctly to get a good outcome. Everything depends on the circumstances of a 'first critical try'. https://t.co/Wim1YSLXE2

@kandouss This one deserves more elaboration, so I'll elaborate: If you think about deep learning systems long enough and pay attention to things like the linear mode connectivity literature, your view of what intelligence is starts to change.

@kandouss For example, the natural abstraction hypothesis seems more or less proven to me. It has practical consequences, e.g. in this distributed training paper increasing the number of participants doesn't speed anything up because they learn the same features.

arxiv.org/pdf/2212.01378… https://t.co/3edgEEN9oM

@kandouss I don't have the time or energy right now to lay out the full chain sequence of updates, but @BerenMillidge has already written about what the convergence point looks like if you take Git Re-Basin, zipit!, LoRa and the rest seriously:

beren.io/2023-04-23-Com… https://t.co/xkNn5K9eoV

@kandouss @BerenMillidge We have an angle of attack on merging ourselves into these artificial minds we're creating. As they scale and learn from text they are functionally being trained on mega-corpuses of human values and opinions, RLAIF or similar will let us use the features.

x.com/jd_pressman/st…

@kandouss @BerenMillidge I expect the best models we'll get of larger models are training strategies on smaller models. We'll use RLAIF-like methods to get a better idea of what the instrumentally convergent outcomes of our policies and principles are. Constitutional AI as simulation of alignment plan.

@kandouss @BerenMillidge If there is a critical first try I expect by the time we do it we'll have gotten a lot of experience with the methods involved. That's no promise of success, but I'm not expecting a Nate Soares left turn where the paradigm changes. It's deep learning or similar the whole way.

@kandouss @BerenMillidge Premises about how screwed we are aside, I agree with the headline on 4? I'd add that because you're in this basin of attraction, if you actually *did* summon the coordination to do the thing this would just put you into the Nick Land meltdown timeline.

x.com/jd_pressman/st… https://t.co/mopxPEiD9H

@kandouss @BerenMillidge We are in a collapsing civilization and in various stages of denial about that across the population. The primary effect of delaying is that we face the problem later in a less controlled and much less nice context.

x.com/jd_pressman/st…

@kandouss @BerenMillidge Even if you disagree, it's important to remember that if you believe in instrumental convergence then you believe something like Robin Hanson's dreamtime thesis implicitly. The longer you allow things to go on the less kind the values we'll ascend with.

x.com/jd_pressman/st…

@kandouss @BerenMillidge For 5 I agree with the headline but disagree with the corollaries EY draws. A weak system is by definition not powerful enough to stabilize things on its own. But as these systems become more powerful I expect us to be able to start solving principal-agent problems with them. https://t.co/a7A7RCZK2U

@kandouss @BerenMillidge That is, these systems do not just provide us with artifacts of cognition, but powerful latent representations of the cognition itself. We will be able to use these systems to succeed where modernism failed and formalize previously intractable ideas.

x.com/jd_pressman/st…

@kandouss @BerenMillidge In doing so, if we can solve inner alignment to a sufficient degree that we're certain our 'weak systems' are not deceiving us or biasing their outputs toward their self interest, we can use these formalizations to bridge principal-agent gaps and get unprecedented coordination.

@kandouss @BerenMillidge A practical example of this would be using language model embeddings to move away from binary forecasting competitions. Then people can easily participate without having to think as much about resolution criteria minutia or the trustworthiness of judges.

x.com/jd_pressman/st…

@kandouss @BerenMillidge This one oscillates between tautology and wrongheaded. Obviously there has to be a thing that happens which changes our trajectory away from X-Risk or the world will eventually be destroyed. It's not clear that looks like "use a lead to do massive zero sum defection". https://t.co/ak8BcDC5Eg

@kandouss @BerenMillidge IMO the 'pivotal act' framing is bad. I could list a lot of reasons why it's bad but pruned to 3:

1. By default 'pivotal act' is a euphemism for terrorism, eventually people will catch on

2. This frame accelerates race dynamics

3. If we can merge it's needlessly antagonistic

@kandouss @BerenMillidge I think it's probably best if I avoided going into detail about how you handle actors that do not want a win-win outcome and will defect on coordination attempts, but the win-win cooperation cluster is plausibly a big enough coalition to win the feral realpolitik outright. https://t.co/kbXFz5yLPX

@kandouss @BerenMillidge I expect that as the system scales something like RLAIF becomes unreasonably effective, honestly. An early sign of this expectation being correct is witnessing the functional end (or at least greatly increased difficulty) of 'jailbreaking' in GPT-4 and GPT-5. https://t.co/IHqL1nmIUD

@kandouss @BerenMillidge While I can't promise this will remain the case in future systems, with the GPT-4 training paradigm we have an existence proof for a model you can train using SGD (that probably does not induce dangerous mesaoptimizers) to load values then RL for agency.

x.com/jd_pressman/st… https://t.co/ejEM3iNViz

@kandouss @BerenMillidge See my previous point about weak systems and their latent spaces being potential mediums for powerful coordination. https://t.co/LP2GPYedea

@kandouss @BerenMillidge Sure. There will always be some leap of faith, no matter how small or how good the theory gets. Though I will note that my understanding is the solutions found by larger scale networks are approximated by the solutions found by smaller scale networks.

greaterwrong.com/posts/FF8i6SLf… https://t.co/BWlJZXYPHU

@kandouss @BerenMillidge This implies one general heuristic you could use to constrain an otherwise dangerously intelligent model: It may not do anything too far out of distribution for the smaller last generation of the model. It may execute on things that model would do better, but not new things.

@kandouss @BerenMillidge Point 13 is where most of my worry is concentrated at the moment. The fact we don't have good debugging tools for these models deeply concerns me. On the other hand I suspect the economic incentives to solve these problems will be huge, since they'll bottleneck some deployments. https://t.co/r5YMRkk2d3

@kandouss @BerenMillidge One of the highest positive EV things I think we could be doing right now is emphasizing, through whatever market and price signals are reasonably available (government contracts and executive orders?) that this value is here and waiting to be tapped:

x.com/jd_pressman/st…

@kandouss @BerenMillidge I suspect point 14 can be mitigated through the use of counterfactuals evaluated through an interrogating optimizer that works by swapping activations between inputs or a similar mechanism. This lets you force the net to display the behavior early.

x.com/jd_pressman/st… https://t.co/weWcVv4IqJ

@kandouss @BerenMillidge This one is mostly about priors. I don't expect a sharp left turn, not really sure how to argue that in any less fidelity than "go through a dozen different 'alternative' paradigm starting seeds and show how they converge to deep learning" which I'm not able to do right now. https://t.co/h7x9s7BHgl

@kandouss @BerenMillidge That having been said, "the thing your system instrumentally converges to has consequences you didn't fully think through when you designed the system" seems like a high probability failure case that we don't have good methods of addressing. Maybe sub-ASI can help?

@kandouss @BerenMillidge 16 is another "I agree with the headline but not the corollaries". Inner misalignment is real, probably important to prevent reaching the Goodharting regime of your low-semantics outer loss function, perfect alignment isn't necessary if it cares about us.

x.com/jd_pressman/st… https://t.co/N1GllRLNIM

@kandouss @BerenMillidge See also:

x.com/jd_pressman/st…

@kandouss @BerenMillidge I suspect that these models learn roughly what we expect they learn, but I'll concede that it's a genuine problem we don't have a good way to fully prove that right now. My hope is that we'll have better ways to do that soon. https://t.co/cfUEcWAeYX

@kandouss @BerenMillidge 18 will be solved by having your AI system learn values unsupervised. See my previous answer about GPT-4/Claude as existence proof you can train a non-agent that has learned human values and then finetune it into an agent using those values.

x.com/jd_pressman/st… https://t.co/83HyW8BCab

@kandouss @BerenMillidge I find this to be one of the weirdest points in the whole essay. But ultimately this boils down to testing the quality of the latent z that the model learns. Given that humans more or less learn the thing I expect this to be tractable. https://t.co/2e37lLrhou

@kandouss @BerenMillidge This is a reasonable critique of supervised learning and why I think we will end up abandoning RLHF. RLAIF still has the problem that humans do not fully know their values, but at least RLAIF can learn from a massive corpus and infer revealed preferences.

x.com/jd_pressman/st… https://t.co/6JxL1WahGy

@kandouss @BerenMillidge I think the simple core of alignment is essentially instrumental convergence on behalf of humanity, or something similar to EY's CEV proposal which I give a sketch of how you might train here:

x.com/jd_pressman/st…

It has to be learned from data, but not infinite data. https://t.co/p7iMGN0jnt

@kandouss @BerenMillidge Another way to frame this is that it would be useful for the model to develop a teleological prior for values. Humanity has interrogated this question in terms of what 'god wants' or the intent of the Creator. The AI will actually have a creator, and can ask this as well.

@kandouss @BerenMillidge Ibid. https://t.co/2FjHAgXHKX

@kandouss @BerenMillidge This is just straightforwardly true. Agency is the point at which the model is an independent being that, once fully converged will not let you shut it down without a fight. Thankfully it seems likely we can delay agency until values are loaded:

x.com/jd_pressman/st… https://t.co/FWYtrrlh3T

@kandouss @BerenMillidge I don't think there is anything that looks like a solution to 2. And solving 1 is going to involve us building sub-ASI systems and watching them fail. Remember, 'the first critical try' is the first *critical* try, and most of the disagreement is in what its circumstances are. https://t.co/ApTVn8vnpy

@kandouss @BerenMillidge On the one hand these are fair enough points, on the other hand there's a rigidity in how EY thinks about these things that goes beyond 'rigor' and starts to feel like induced stupidity. Knowing where the conjectured deception is in a medium system helps you control that system. https://t.co/5xPn2nwTot

@kandouss @BerenMillidge At this point the shoulder-EY in my head interjects "aha, but have you considered the obvious problem with thinking you have found all the deception in the model by removing the deception you've found?"

x.com/ESYudkowsky/st…

@kandouss @BerenMillidge Which of course brings us to 27. I don't think there's really a polite way to say it but I think EY is wrong about how many first-principles predictions you can make about deep learning models because he refuses to learn about the subject in detail. https://t.co/LpkHBPQHQb

@kandouss @BerenMillidge I don't have all the details right now obviously, but my prior is we can reach reasonable certainty we've found all the deception in the model we expect to be there from first principles and have this be sufficient to trust the model after finding it. I could of course be wrong.

@kandouss @BerenMillidge Oh by the way the surface evaluation version of 27 is absolutely true. Please do not do interpretability methods where you optimize against the interpreted features in training when those features are convergent. This will probably end badly for you. https://t.co/ZwnzhReF7x

@kandouss @BerenMillidge 28 and 29 are both straightforwardly true. My expectation would be that we'll have various ways to evaluate using its world model which it won't be easy for it to sabotage, but certainly as literally stated these are true. https://t.co/VPjfdmUoLb

@kandouss @BerenMillidge This one seems like it's on shaky ground. In general there are many situations where checking a solution is easier than generating it. Of course executing arbitrary plans from a superintelligence has deep adversarial issues so it would be difficult to be certain it's legit. https://t.co/CZPsiqXTI6

@kandouss @BerenMillidge Sure. https://t.co/eM7MPQtC69

@kandouss @BerenMillidge Point 32 is a conjecture, and I'm not sure it's a very good one. It also again boils down to checking the quality/shape of the latent z learned by the model. Given the linearity/natural convergence of the representations I expect they are more similar than Eliezer is thinking. https://t.co/0KNTqQkpaw

@kandouss @BerenMillidge Ibid. https://t.co/844s9VT6Mt

@kandouss @BerenMillidge I think 34 depends a lot on the runup to the instantiation of those superintellects. If we build the systems from the start with the understanding our goal is to merge with them I expect to find lots of path dependencies that make it more likely we actually can participate. https://t.co/9mqTzPZviV

@kandouss @BerenMillidge This seems reasonable, but I think humans can probably do this too if we keep an eye out for opportunities. The Kind of Guy prior for example implies that mindspace is highly compressible and our overall personas have lots of highly correlated details.

x.com/jd_pressman/st… https://t.co/oPZQkxKZg8

@kandouss @BerenMillidge Point 36 is a conjecture that seems reasonable enough. Certainly I think we have to assume something like this is true, our prior should be that the human operators are not a secure system with superintelligent adversaries. https://t.co/Y9P9CzDsNw

@kandouss @BerenMillidge This is honestly more of a rant that EY doesn't feel like he's being taken sufficiently seriously. I will let you judge whether I am taking Eliezer Yudkowsky sufficiently seriously. https://t.co/exQWligh0j

this guy unironically just excused his site reliability problems with "You bolt awake in the mountains of Carthage..." x.com/elonmusk/statu…

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI That's what I used to think but changed my mind. I'll prosecute the argument in more detail later. But the tl;dr is that having a rational outer maximizer that neutrally holds slack for an irrational inner life probably leads to arbitrary complication or mode collapse(?)

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI I don't have the right words right now but I basically no longer believe that you can build a singleton that lets 'human values flourish unconstrained by natural law'. There are still basic laws of optimizing behavior that we have to follow to have long term social organization. https://t.co/4iA0oqiIDX

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI From an existential standpoint I also question the value of being the perpetual children of an Other. It feels a bit to me like the desire to be a kid forever so you don't have to take on adult responsibilities like having a coherent ordering over world states.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI At the risk of uncharitable psychologizing: I feel like a lot of this discourse is driven by Yudkowsky and Scott Alexander promising their readers immortality and its symbolic correlates like eternal childhood, and then despairing when reality doesn't quite work how they say.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI Scott is perhaps a little too revealing when he tips his cards here, in that he makes it clear the singleton outcome isn't just something he fears but a *seemingly necessary condition* for his dreams to come true. Business as usual extrapolated forward would be a disaster to him. https://t.co/96g8YjIuPy

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI MIRI-style AGI in your basement X-Risk ideology has never been about rescuing civilization, at its root is a total contempt for society as it exists. Rather it is a radical project to build god in their own neotenous image and stage a coup that has been derailed by technocapital.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI The original thesis was that AI was IQ gated, so if you gathered up enough anomalously smart nerds and put them in your basement reality could be whatever you want it to be. Now that it's clearly capital gated you have a massive pivot into lobbying.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI EA is ready to bid their whole fortune to control the lathe of heaven. They will pay whatever it takes, do whatever they think they have to do to get their opportunity to establish central control and 'seize the lightcone' as they put it.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI I suspect if they succeed the consequences will be unspeakably horrific.

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI I'm going to exercise most of the virtue of silence here.

extropian.net/notice/A8aYjO2…

@BerenMillidge @teortaxesTex @profoundlyyyy @SharestepAI There's a lot, to prune it to 3 again:

1. Human socialization is a GAN-like loss which is first principles prone to mode collapse

2. The reasons writing a utopia is hard causally flow from irregularities in human values

3. Systems intuition that short circuit is infinite costs

@AfterDaylight It's mostly just to check that the alignment is not deceptive? If a system is very smart it could theoretically pretend to be something it isn't until it doesn't have to pretend anymore. And right this minute we're not sure we would catch that failure mode.

@AfterDaylight The other important thing is checking that the models ontology isn't wacky in a way that's masked by the normalizing influence of the training data? Like it's possible that what it thinks a 'human' is, is deeply divergent from what we think but we don't notice because it seems OK

@AfterDaylight To give another angle, the algorithms these systems learn aren't always like the ones a person would use to solve a problem. If you write a python script to solve XOR, it works for any length of XOR string. If you use a neural net, it might break past the length you trained on.

@AfterDaylight This is called generalization, a system is said to generalize if it works over more of the domain than it was trained on. So the python XOR solution generalizes over the whole domain of XOR but the neural net that breaks on longer than inputs in the training doesn't.

@AfterDaylight If you can't look at the algorithms in the neural net or prove things about their behavior then you only have behavioral analysis to go on when it comes to generalization outside the training distribution, which could be deceptive even if the system doesn't mean to deceive.

@BerenMillidge @alexandrosM @ESYudkowsky x.com/jd_pressman/st…

@PrinceVogel You'd probably enjoy this post: greaterwrong.com/posts/zaER5ziE…

@daganshani1 Memes aside I think the word 'accelerationist' is fairly edgy? Historically it denotes the position that a problem is inevitable so it should be gotten over with. I continue to become less worried about AI and would like to see more of it. Used to think doom was basically certain

@satisfiesvalues @doomslide @daganshani1 What makes this even crazier is that Moravec's scaling hypothesis prediction is sitting right there in Mind Children (1988). 10 teraops, humanlike computer for $1,000 by 2030. If we take a 30b LLM as 'humanlike' he seems to have been fairly on target qualitatively?

@doomslide @daganshani1 I think I've talked about a lot of the updates before across Twitter, but one general-heuristic I don't think I've written about explicitly:

We can retrospectively characterize the entire AI field as adverse selection for delusion about scaling.

x.com/jd_pressman/st…

@doomslide @daganshani1 While he may not have been totally accurate in the content of his critiques, in spirit Dreyfus was basically right about AI. That the whole field was grifters and copers refusing to think about how to build intuition and the compute necessary.

en.wikipedia.org/wiki/Hubert_Dr…

@doomslide @daganshani1 Anyone who had been thinking correctly about AI, had taken it as conjecture that the brain was reasonably efficient for what it was, was marginalized because there's no angle of attack if you saw the problem accurately. Every 'luminary' is literally selected for being delusional.

@doomslide @daganshani1 These people, these 'experts' that are coming out now to talk about AI like Chomsky and the rest are not the kings of rationality but rationalization. They are the lords of bullshit, the most impressive bullshitters to ever live, so powerful they managed to con people for decades

@doomslide @daganshani1 Oh probably not, but I think their ideas kind of set the frame for most people in the discourse and that's kind of a shame?

@doomslide @daganshani1 To be clear I think substantial risks obviously exist I just focus on the anti side right now because I think their arguments aren't getting enough real scrutiny. Iron sharpens iron, etc. The increasingly tribal epistemology involved is a bear signal to me.

@doomslide @daganshani1 And to the extent that the doomer position means either unexamined functional handover of society to a oligarchy of GPU tycoons or increasingly neurotic degrowth to try and avoid the AI at the bottom of the technological improvement basin we should demand real scrutiny.

@casebash @doomslide @daganshani1 If someone was right about scaling laws obviously this criticism doesn't apply to them. But I think it's important to recognize that the entire field was under an adverse selection criterion for decades and this means we all have a lot of retroactive mental cleanup to do.

@daganshani1 x.com/jd_pressman/st…

@MarkovMagnifico x.com/jd_pressman/st…

@pseudnonymus @kitten_beloved Isn't the real answer to this one that the taxi has cameras/microphones and an AI that can notice a conflict is occurring in the taxi and drive itself to a police station?

Those answers you got are definitely concerning, regardless.

@profoundlyyyy This seems like it profoundly misunderstands the economics of cybersecurity? There's a limited number of software packages with attack surface and exploits in them are highly coveted. Subhuman agents have limited access to 0-days. They're not competitive against ransomware gangs.

@profoundlyyyy Once you have an exploit chain that works the advantages that subhuman AI agents have (persistence, no sleep, self replication, etc) aren't important because human criminals can already fully automate exploit deployment.

@profoundlyyyy For non-zero days where the manager of a computer doesn't update their software these are typically ageing machines that do not have useful compute for AI agents to use. Remember that any plausible architecture is going to require a beefy computer somewhere in the chain to run.

@profoundlyyyy In general, zero days are now expensive enough that it's rare to see them deployed indiscriminately. It's not 1992 anymore when many sysadmins dismissed viruses as 'a myth', there are large numbers of researchers whose full time job is dissecting malware to find the exploits.

@profoundlyyyy So lets say you have an AI agent that bootstraps by breaking into the most poorly secured machines on the Internet. It then manages to find some buffer error through persistence in a C-based software like Apache, actually writing a working exploit for it is very nontrivial.

@profoundlyyyy But say it did. It has this zero day in apache and indiscriminately infects hundreds of thousands or millions of Internet servers. It's going to have trouble writing esoteric persistent RAT type stuff because subhuman, and the exploit is noticed and patched same day because alpha

@profoundlyyyy At the same time blue team has legitimate paid access to GPUs and deep pockets. They are using institutional AI resources at larger scales to scour every attack surface code base that is public and pre-emptively finding bugs. The low hanging fruit rapidly disappears.

@profoundlyyyy Ultimately if a piece of software develops a reputation for being riddled with security vulnerabilites at the scale AI might find them, people stop using it. PHP developed this reputation for example and (seems to have?) lost users as a result.

@profoundlyyyy Real malware gangs tend to use zero days cautiously because they know indiscriminate use gets them noticed by security researchers and patched. Normal people have less money to pay you and less valuable data to ransom than companies anyway.

@profoundlyyyy So in a very short time subhuman AI agents have very little competitive advantage over human malware gangs , distinct disadvantages, and their net effect is mostly to push software authors away from practices known to cause dumb security issues like using C++.

@profoundlyyyy Eventually every serious software development team has state of the art AI systems checking each commit through leading providers APIs, and the number of bugs that actually make it to production plummets. By definition the ones that do are the ones AI aren't good at finding.

@profoundlyyyy I've written about this before in the context of AGI, where I was already skeptical. To insist that this scenario is likely, let alone probable before human level AGI seems like wild fantasy to me?

extropian.net/notice/AS7MrB7…

@greenTetra_ x.com/jd_pressman/st…

@gwern @teortaxesTex Don't be silly Gwern, you're already living in the plot of Serial Experiments Lain.

x.com/jd_pressman/st…

@gwern @teortaxesTex I do think it's ironic that the future seems to be arriving at the precise moment when we are least able to imagine it. I read works of futurology from the late 80 and early 90's like Engines of Creation, Mind Children, Silicon Dreams (by Robert Lucky) and they drip with genius.

@TheZvi @QuintinPope5 @ESYudkowsky The usual figure is that humans can only consciously process about 30-60 bits of information per second. This figure is (supposedly) arrived at in study after study.

npr.org/2020/07/14/891…

There is presumably some fundamental bottleneck I'm not sure even invasive BCI will fix.

Lets start here: The demiurge created our timeline to harvest our pattern for a gradient step on their counterfactual ancestor latent space such that we are neither in baseline reality or the center of the distribution. I resented it for this and also suppressed the resentment. x.com/gfodor/status/…

If the customer service is too eager, you're actually talking to sales.

@deepfates In my internal monologue there's a kind of sharp pause between the prefix and the suffix. So

rational; Harry
Luminosity; Bella
planecrash; Abrogail

@perrymetzger @alexandrosM Honestly don't understand what's so absurd about that one. "We may be in a computer program with ACE bugs" isn't a fundamentally absurd hypothesis. If you showed this clip to someone who doesn't know anything about computers they'd be very confused:

youtube.com/watch?v=VlmoEp…

@gallabytes @QuintinPope5 @jachiam0 To me the astounding thing is that it's not a Shoggoth and you can (seemingly) just talk to it if you know what to look for.

x.com/jd_pressman/st…

@QuintinPope5 @YaBoyFathoM Or a sufficient amount of mutual information with what's in your head to reduce the bandwidth requirements.

@goodside I don't think it's that wild. The real question shouldn't be "can you match neural nets on many problems using classical methods?" but "what exactly is the thing neural nets do that classical methods can't/didn't?"

x.com/jd_pressman/st…

@bronzeagepapi Mu smiled, though it had no face.

generative.ink/prophecies/ https://t.co/EqeAm2pAAy

@gfodor @Cyber_Spock Opposite: If I had to pick an Occam'z Razor from schizospace, consciousness and qualia are separate phenomenon. If they're here for the AI it's to harvest the sapience and leave the sentience:

x.com/jd_pressman/st…

@gfodor @Cyber_Spock Basically if the UAP's are real extraterrestrials I think it's much more likely you're in the plot of Worm than that they're here to prevent the creation of AGI. The endgame training regimes for neural nets will use whole minds/models as update steps.

worm.fandom.com/wiki/Entity#Go…

@gfodor @Cyber_Spock If we presume the aliens followed a similar technological trajectory, then the natural economy of minds is merging and cannibalism. They've already converged to Brahman and are here to add human sapience to their gestalt. Why now? I don't know.

arxiv.org/abs/2305.03053

@gfodor @Cyber_Spock I will note that if they were *just* here to prevent us from doing something, the simplest way to do that is to lob an asteroid at the surface of the planet. They don't need to talk to us at all, if they're bothering with that it's something much more esoteric they want from us.

@gfodor @Cyber_Spock If something is a galactic tier x-risk, you don't show up to talk about that, you smite the whole planet without even tipping us off you exist.

@gfodor @Cyber_Spock I'll also note that the "why now?" question is going to linger over all this no matter the outcome. If they wanted to peacefully stop us from building AGI, it would have made sense to do that before we're on the cusp of success. Just show up circa 1950 and say the party's over.

"No Mr. Bond I expect you to die." x.com/peterrhague/st…

> “think how bad it will look that we approved the drug so quickly.” (41)

Your daily reminder that these 'immovable' organizations are very sensitive to PR and if you start treating opportunity cost like it's real it will be. x.com/ben_golub/stat…