Add question about AI usage to PR template by ekzyis · Pull Request #2507 \ stacker news ~devs

pull down to refresh

Add question about AI usage to PR template by ekzyis · Pull Request #2507 github.com/stackernews/stacker.news/pull/2507

I recently added this question to our pull request template:

Did you use AI for this? If so, how much did it assist you?

I am quite excited to see how contributors will answer this question.

It is not meant to judge them if they do use AI, but it is meant for us to better know what to expect from a PR.

I'd also rather not make snarky comments like this one, but sometimes, I just can't not express at least to some degree how I feel about something. If they are just upfront about their usage, then I don't have to guess anymore.

Also, it's also a nice way to see if people are honest. Sure, they can lie, but if I ever catch them, oh boy, I will not hold back. To be honest, lying in a pull request might even be grounds to get banned from the repository as far as I'm concerned.

In theory, the answers could also be used in a case study.

view all related items

100 sats \ 4 replies \ @optimism 7h

I've always taken a position that it is less important if someone used AI than that they understand and can maintain the code they are proposing. So I never directly ask "did you use AI?" but instead I ask questions about choices made. That's my way of "being nice"

Speaking of being nice...

I'd also rather not make snarky comments

That one is not too bad, imho. I tend to make grown men cry when they come with their grok/gpt assisted bullshit on my repos. I also have a lot of enemies, so don't be me.

100 sats \ 3 replies \ @ek OP 7h

instead I ask questions about choices made

I also did this but I came to the conclusion it’s a huge waste of time if they don’t just say they don’t know because the AI came up with it.

I just want them to be upfront so I don’t have to guess anymore if I’m about to waste my time thinking about details of a PR when it turns out it was just AI, nothing deep going on

521 sats \ 2 replies \ @optimism 7h

I liked this policy proposal on forgejo.

In general, I glance over every pull request and I prioritize by cleanliness. I did that before AI too: if you throw slop of any source over the wall, then you gotta wait.

100 sats \ 1 reply \ @ek OP 6h

Thanks for the link, very interesting!

For example, I hate reading stuff like this, lol:

FOSS projects that have accepted the use of AI contributions (or integrated AI features) have not been received well within the FOSS community

Why does it matter how something is "received"?

This is a technical discussion, it should be based on technical merits.

And usually, "community" just means social media, not sure how many of them are really FOSS developers.

But I guess I should understand that it makes sense to mention. (?)

100 sats \ 0 replies \ @optimism 4h

I hate reading stuff like this

Me too, but in this case I mostly ignored the hippie rationale. The basic rules proposed make sense, i.e. proposed rule 1-4. Emphasis mine:

If content was made with the help of AI, you must convey that this is the case. This includes content that you authored but was motivated by a suggestion of AI.

This is basically the same thing you propose.

If at any point you used AI's work in your contribution you must demonstrate that you can submit this under the license of the repository.

This means you likely can't use code generated under another ToS (i.e. that of Claude or GPT), but this is a non-AI thing too, so it should technically not be AI-specific.

The accountability of using AI in a contribution lies by the person that makes that contribution.

Very nice liability limiter, but shouldn't this be the case on all code?

All communication, that includes: commit messages, pull request messages, documentation, code comments and issues (and comments on issues/pull requests), that is intended to be read by people to understand your thoughts and work must not have been generated with AI. We exclude machine translation and tooling that helps with grammar and spelling check.

This is imho the most important point socially. If you want me to read something, make it so that I want to read it, but again: this is not AI-specific. ¹

The other items are not something I'd encourage on a non-niche project, but I guess it's fine for those among us that want to cater not to the world, but to their faction. You have to trust humanity to develop for the world.

I've for a longer time been playing with the idea to improve contribution guidelines based on something like this, but I keep getting back to the point that AI should at most be an example for illustration, not a rule.

Why does it matter how something is "received"?

I do think that they have a point as many people that move away from GH nowadays do so because of copilot skepticism - GitHub went all-in on AI, so the alternatives should be careful. The AI fearful/skeptics need a "home" too (though I'd argue it's much nicer to be homeless / squatting across all platforms, and this also helps against cults.) ² But bottom line this is more marketing / politics.

"community" just means social media

Agreed. Especially since on many of these platforms you're not allowed to comment on an issue "wtf is this shit", because that will get you instabanned for life, lol.

I've for a longer time now been blessed with proper CI on my repos (after I spent years to have that) so I have largely switched my mode in day-to-day review to spend a lot of time on comment and commit quality. It is not for a reason that on my projects I am the most hated maintainer (but also the most senior so tough luck lol) ↩
Unfortunately the politics of many of these alternative platforms, including codeberg, are very inclusivity-and-safety focused - I wouldn't run a meaningful FOSS repo under their CoC, because those rules can be weaponized and selectively enforced. ↩

55 sats \ 2 replies \ @south_korea_ln 8h

Lol, i should ask my students the same question... So much slop

40 sats \ 0 replies \ @SimpleStacker 4h

On my syllabus I explicitly reserve the right to challenge their understanding of any assignment with an oral assessment if I suspect them of having used ai without understanding

My general policy is you can use it as long as you understand what you wrote down and you're not copying verbatim

0 sats \ 0 replies \ @ek OP 7h

Yes, why not?

100 sats \ 0 replies \ @SimpleStacker 12h

It might be helpful to have some kind of rubric for levels of AI assistance. Off the top of my head it could be like

No AI assistance
AI was consulted for help, but did not write any of the code
AI provided tab-autocompletions while coding
Significant sections were based on AI first drafts
Vibe coded (entire first draft written by AI)

100 sats \ 0 replies \ @adlai 13h

In theory, the answers could also be used in a case study.

There are statistical techniques for blinding the answer to embarassing survey questions. Here's the simplest example:

Let's say your research is funded by the psychology department, and aims to uncover deep truths about the human mind... however it must compete with hunger and tiktok, so you have a budget and compensate your participants. The first problem is a simple one: is your participant motivated by the compensation, or purely altruistic and simply responding to your survey out of the will to improve everybody's understanding of the human mind? It is arguably the least embarassing question, use your imagination for the racier ones psychologists might ask...

The method goes like this. You give the participant a fair coin, and allow them to flip it enough trials to convince themselves of its fairness. You then tell them to flip the coin twice, and answer the following compound question:

If the first coin flip was heads, then answer the question of whether you're motivated by the survey purpose more than the compensation; otherwise, answer whether the second coin flip was tails.

Each individual respondent now has one bit of entropy blinding your knowledge of their motivation, however the actual possibilites are a predictable normal distribution when you have a statistically significant number of respondents, and knowing the distribution of lots of coin flips, you can even calculate the confidence bounds that you obtained from a given pool of respondents.

17 sats \ 5 replies \ @DarthCoin 10h

0 sats \ 4 replies \ @ek OP 10h

I thought you don't know much about code

21 sats \ 0 replies \ @SimpleStacker 4h

He knows enough to know that ai coding is still pretty jank haha

0 sats \ 2 replies \ @DarthCoin 10h

I understand a little bit but I will never use that book. I post it just as a joke.

0 sats \ 1 reply \ @ek OP 7h

95% of my replies to you are also just a joke / trolling

Thought you would know by now lol

0 sats \ 0 replies \ @DarthCoin 7h

Footnotes