• naeap@sopuli.xyz
    link
    fedilink
    arrow-up
    9
    ·
    12 天前

    Maybe it’s answered in the article - but I’m just jumping into bed…

    How will they differ between AI created code vs “natural” code?
    I do know, that there are some give-aways - can’t put my finger on it, but you can see it’s AI generated
    But with many contributions, I’m not sure how they would be able to detect it

    Or is it about using AI in other ways?

    • tangeli@piefed.social
      link
      fedilink
      English
      arrow-up
      17
      ·
      12 天前

      Even if AI generated code can’t be detected reliably, it is good to have a clear policy to guide projects and contributors. At least those contributors acting in good faith and honestly will do the right thing, and projects will have a basis for rejecting contributions from people that are found to willfully or negligently violate the policy. Blocking all malicious actors will be an ongoing effort, whatever the policy.

      • radioactivefunguy@piefed.ca
        link
        fedilink
        English
        arrow-up
        5
        ·
        11 天前

        For me, if a PR looks good, and I can reach out to the contributor and they can competently discuss their work, then its fine with me if they used AI to code it.

        I think a sensible policy would be: use whatever tools you like to contribute, but own the contribution. Obvious slop code is rejected.

        • tangeli@piefed.social
          link
          fedilink
          English
          arrow-up
          8
          ·
          11 天前

          I agree as far as responding to questions and issues that arise from the contribution but my understanding is that from a legal, copyright perspective, if AI generated the code then the contributor can’t own the copyright, nor assign it. So my preference would be: no AI generated code. Other use of AI tools (e.g. finding vulnerabilities) might be OK but there are still the environmental and ethical issues to be concerned with, so I remain ambivalent. It will be interesting to see the decision of the Debian community.

    • ɔiƚoxɘup@beehaw.org
      link
      fedilink
      English
      arrow-up
      6
      ·
      11 天前

      I’m just jumping out of bed, but I’ll give you a quick answer. You’re right, in some cases it’ll be impossible to tell that code was AI-generated, but what you’ll have is the reputation of the developer. If any of their other code is AI-developed, then they would be suspect, right? So, I think it’s gonna be something like that, and, like with other things in the open source community, a matter of trust and respect.

      • naeap@sopuli.xyz
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        10 天前

        Sounds more like detective work, than a coffee code review
        This will cost quite some time for the maintainers :⁠-⁠\

        • ɔiƚoxɘup@beehaw.org
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 天前

          I mean, maybe it could be that, but what I think it’s going to end up being is more like a reputation thing, like an actual community with standards and basis for trust with vetted individuals that have a reputation of producing good code by hand, and testing it correctly.

          Think of it this way. There’s already a standard practice for keeping people that suck at coding from trying to write code, right? I trust that the same community norms that got us this far will continue to do well. I also believe that there will be some that have reputation issues because of AI.

    • Jul@piefed.blahaj.zone
      link
      fedilink
      English
      arrow-up
      2
      ·
      12 天前

      There are “signatures” that can be detected in code just like universities detect it in student work, online stores detect it in reviews they compensate people for, and some employers detect it in resumes and cover letters. It’s bland, just like when it tells you to remove colorful phrases or adjectives that add emphasis from your text in an email. Things like that.

      That said it doesn’t detect it all, and it does rely on the developer to be honest. But it’s also a good indicator when reviewers post questions to the developer and they can’t answer them because they don’t understand the code and things like that. Also, if the developer did not follow coding standards, which LLMs can sometimes have trouble with. And there’s always been malicious code submissions in open source software, so there are other ways.

      Anyway, it’s more of a policy discussion at this point, and why LLM generated code is bad. I use them for manipulating data, figuring out syntax, and getting examples, but there are always problems with anything they generate, so I never would allow one to write any production bound code. I see use in the technology, but it’s not creative and it doesn’t understand what it’s writing so it can’t write code with consequences in mind. So it can write Jr level code, but not solid code, at least not without being properly trained on only well written code, and that proper training would be too expensive for an LLM company to actually do it.