I really feel like we had several prominent FOSS organizations fail us by not saying "hey hold on, don't use generative AI in FOSS projects, we're not sure what the legal, community, environmental implications of this are but there are serious problems and we have a history of taking a stand" and I worry that maybe we lost our moment to be principled, except in niche spaces, and the walls are closing in

And... I'm really sad about it. I think we're underestimating the amount of damage that's happening. :\

I don't think you can put all else aside, but even just focusing on the maintainability of code itself...

Using these tools to uncover bugs, sure. Using them to generate code that nobody in your community fully understands... well...

And I know, you say, you'll be responsible about it.

But you won't be able to keep up, justifying keeping up with that level of review won't happen. So you'll slip. And you'll eventually embrace not knowing. And so is everyone else around you.

And so, decades of work are knotting up and the foundations are ensaring beneath our feet.

And we could have, collectively, organizationally, taken a stand, and we didn't.

@cwebber

And so, decades of work are knotting up and the foundations are ensnaring beneath our feet.

Having been inside various Linux kernel subsystems and also various fundamental userspace components, I’m not convinced our feet aren’t already ensnared in branches covered in thorns.

@cwebber I think we disproportionately skew towards people who work on artisanal, crafted software. Things that are mixtures of artistry, human creativity, and masterful engineering.

But Commercial Software is not, in general, like that, being subjected to the forces of the market, and the bits of it which retain those qualities are the results of engineers vigilantly guarding it.

Sometimes at work I think about how I can impart my knowledge, preferably in the form of documentation, so that future engineers who come after I leave the company can understand the design and how to add different types of extensions without twisting the thing into knots.

But to a large extent this is hubris, and the human desire for legacy. It is an action for which I have zero economic incentive. I will never, and can never, be judged for it. Even in realms of backchannels people are only punished if a system is obfuscated out of malice.

And much open source that is “important” is fundamentally commercial software done in the open¹. All those same forces apply to it. In some ways its worse, because a lot of the externalities are pushed onto people outside of the employer of a given contributor.

LLMs do not write artful code, but lots of code is not artful; it is moderately to majorly ugly, full of historical warts and design defects that could use a refactor. They can sometimes make that unartful code significantly more malleable. They can make us forget things, but on sprawling codebases it is also sometimes questionable whether we really knew something; and on the flip side, they can extract knowledge from codebases at rates humans just can’t, which makes it possible for humans to acquire contextual knowledge they just otherwise wouldn’t spend the time developing.

Its really weird how they change what things are easy and what are hard. It’s really weird the ways in which they are simultaneously skilled and awful. I am continuously in work code review asking some coworkers to delete the beyond useless comments that Claude has written, while on the flip-side an LLM can often do a tedious refactor in 3 minutes of engineer time that would have otherwise taken 3 days.

And watching things from a work perspective, it’s also really interesting how my relative opinion of various longer-term coworkers mostly hasn’t shifted, but mostly that for the ones I had less respect for my respect has degraded further.

It’s… a fascinating thing to watch play out.

–

¹ The other categories are really (a) a small number of relatively-artisinal projects by small teams; think curl, sqlite and such; and (b) an author or two’s pet projects. You might say there’s (c) basically unmaintained, but that’s also lots of commercial software!

@erincandescent
> But to a large extent this [drive to document the systems I work on] is hubris, and the human desire for legacy. It is an action for which I have zero economic incentive. I will never, and can never, be judged for it. Even in realms of backchannels people are only punished if a system is obfuscated out of malice.

I experience that drive; those are not the only options.

Rather, I empathise with those who maintain the system after me.

@cwebber

@bignose @cwebber there are ways in which it is reasonable and ways in which it is hubris.

My co-workers understand the system perfectly fine and they understand how to extend and modify it. Sometimes they make decisions on how to extend it that don't match my ideas, but they're probably valid. It becomes their garden to tend, not mine.

So there's an extent to which that documentation is actually valuable and there's an extent to which it's just about hubris and legacy and an inability to let go of our children despite the necessity of doing so.