• 1 Post
  • 354 Comments
Joined 3 years ago
cake
Cake day: July 29th, 2023

help-circle
  • Article on businessinsider with the nauseating title “The modern parenting throuple: Parents are at a breaking point — AI companies see an opportunity” because the age-old silicon valley product design technique of asking what your mother used to do for you, and how it could be sold back to you via a subscription is still very much in use.

    https://www.businessinsider.com/parents-breaking-point-ai-companies-opportunity-2026-8

    I confess I haven’t read the full article (which is why I haven’t given it a proper post) but hypervisible.blacksky.app has kindly pulled the highlights:

    https://bsky.app/profile/hypervisible.blacksky.app/post/3mtupdpfcjs2p

    One of the best bits:

    The most tech-savvy parents have built their own tools. Sarah Baldeo, a cognitive neuroscientist, spent the past year traveling for work, so she built a custom GPT of herself to chat with her 15-year-old son when she’s unavailable. Baldeo tells me she programmed the chatbot with her personality, uploading screenshots of texts between her and her son, along with documents about her values, the family rules, and challenges she felt in parenting.

    Baldeo wanted her son to ask the chatbot advice, and he did prompt it with questions he may have fielded to her, like, How do I diplomatically tell my teacher I’m overloaded with assignments? But he also asked the mombot how to convince his human mom to let him hang out at a coffee shop in Chicago in the evening — something Baldeo had been hesitant to allow.

    The bot told him, as Baldeo recounts, “Your mom very much values new experiences, being around people from different cultures, different socioeconomic backgrounds, and this is how you should frame it to her: that you’re exploring a part of Chicago that is more diverse.” Her son took this tact to Baldeo, and she said yes. It was only later that she saw the chat history that let her know he had gamed the bot to manipulate her.

    🤮

    Why the fuck do these people have children. Why the fuck does any functional human being think it is a good idea to outsource raising their children to sam altman and his chatbot.

    Who on earth thinks that training a chatbot to wear their personality like a rubber mask and manipulate people is going to end well? Honestly, I think this is perhaps the bit that triggers the most revulsion in me. I cannot comprehend wanting to be complicit in my own replacement by this thing, but so many people seem excited by the prospect of doing this to themselves.



  • New book: superlawyers! how claude makes you the best lawyer in the room.

    https://www.amazon.com/dp/B0GHYS5FC2

    The author even shares his prompts for free!

    https://github.com/danlorry/superlawyers-prompts

    Every prompt here is designed for that pattern: Claude multiplies your judgment, never replaces it. The prompts are ready to paste into Claude. The book explains why they are structured the way they are.

    The prompts mostly seem to be structured as “imagine you are an attorney. Now do this homework exercise”. There’s a passingly interesting idea around having the chatbot attempt to form a counter argument to your own claims, which you can then use to make your own position stronger, which seems like it might actually be a genuinely okay idea especially if you expect your opponents to be llm sock puppets, but everything else looks pretty much like garden variety “do my thinking for me”.

    I’ll admit I didn’t read the prompts closely, so may be I’m being uncharitable, but I doubt there are many nuggets of gold hidden in there.

    h/t to rahaeli




  • Nah, that’s just the global-warming-induced warm water they like, the presence of a reactor was incidental… swarms around any sort of suction system are going to be a problem.

    I can offer you a upgrade in the form of an array of radioactive spinning blades if you’d like.

    (in theory you could deploy your data centres in places where nutrient levels are low, so there’s less plankton and so a much reduced risk of jellyfish swarms, but I have absolutely no faith that these cowboys would even stop to think about that for one moment)










  • Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.

    There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.



  • And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasn’t been so much of an issue til now because theorem provers haven’t had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.

    https://mathstodon.xyz/@TaliaRinger/117005740997367321

    My big worry right now is that if organizations continue to fund the crap out of Al for formal proof research (and to generally support implementation and maintenance of proof assistants like Lean as part of that effort) but don’t bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.

    Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isn’t obviously mechanisable right now and won’t yield any clickbait headlines?



  • As an interesting follow-up to the ai-does-maths-using lean4 stubstack comments on Sunday, an llm accidentally uncovers a bug in the lean4 kernel.

    Summary by Meven Lennon-Bertrand:

    https://lipn.info/@mevenlennonbertrand/116997917683191056

    To summarize:

    • an AI agent let loose provides a sorry-free proof of the Collatz conjecture
    • the proof is detected as actually being a kernel bug
    • the bug is related to (nested) inductive types, for which there is no clear theoretical specification: the kernel’s code is the reference
    • external checkers (lean4lean and nanoda from a week ago) reproduce the bug, because they essentially copied the reference kernel implementation

    Eta: “sorry-free” in this case means a complete proof with no trust-me-bro steps or TODOs… the sorry tactic in lean “proves” a theorem to be correct even if it is garbage or incomplete. More programming languages should make developers apologise for half-arsing their work.

    And so

    • AI raises the bar for kernel correctness by a lot
    • without a clear type-theoretic understanding of what is actually implemented, we’re toast
    • external checkers help to catch implementation bugs, but without a clear specification they can’t catch logic bugs

    How bad this is, is unclear just yet… probably not the sky actually falling, but not great. Interesting though.


  • I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.

    One of the things that’s never clear from the reporting on ai successes is exactly how much actual paid human time went in to achieving those successes. This was especially notable in the fable-based security work… a huge amount of person-hours went into turning fable-detections into actual meaningful vuln reports.

    A lot of demonstrably clever and capable people are involved with the llms-for-maths work, and a lot of money was spent on their time and supporting their work. Replicating it without your own stable of mathematicians and computer scientists and all the tokens they can eat is probably impractical.

    I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.

    Fwiw, lean is a general purpose programming language, though despite microsoft’s efforts no-one uses it for that. I think its popularity with mathematicians came as a bit of a surprise.

    Anyway, the other important thing that didn’t get reported on is that building the formal definition of the problem is not trivial! Obviously I don’t need to tell you that, but from the reporting you’d think that an llm solved all these problems, when in fact it was an llm in the hands of some very capable people who absolutely did not just prompt the thing in plain english.

    Anyone hoping for self-marking homework here is going to be disappointed… lean slop confirming to formal spec slop is just expensive slop. Reviewing regular genai code is awful, even the thought of reviewing genai dependently-typed code makes me want a new career.