An update from GitHub: https://github.com/orgs/community/discussions/159123#discussioncomment-13148279
The rates are here: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
- 60 req/hour for unauthenticated users
- 5000 req/hour for authenticated - personal
- 15000 req/hour for authenticated - enterprise org
Probably because of AI agents. This is why we can’t have nice things.
RIP yocto builds
Good thing I moved all my repos from git[lab|hub] to Codeberg recently.
did your project manager or client ask for you to move there?
Maybe charge OpenAI for scrapes instead of screwing over your actual customers.
Github is owned by Microsoft, so don’t worry, it’s going to get worse
I honestly don’t really see the problem here. This seems to mostly be targeting scrapers.
For unauthenticated users you are limited to public data only and 60 requests per hour. And for authenticated users it’s 60k/hr.
What could you possibly be doing besides scraping that would hit those limits?
60 requests per hour per IP could easily be hit from say, uBlock origin updating filter lists in a household with 5-10 devices.
You might behind a shared IP with NAT or CG-NAT that shares that limit with others, or might be fetching files from raw.githubusercontent.com as part of an update system that doesn’t have access to browser credentials, or Git cloning over https:// to avoid having to unlock your SSH key every time, or cloning a Git repo with submodules that separately issue requests. An hour is a long time. Imagine if you let uBlock Origin update filter lists, then you git clone something with a few modules, and so does your coworker and now you’re blocked for an entire hour.
I hit those many times when signed out just scrolling through the code. The front end must be sending off tonnes of background requests
This doesn’t include any requests from the website itself
This going to fuck over obtanium?
I see the “just create an account” and “just login” crowd have joined the discussion. Some people will defend a monopolist no matter what. If github introduced ID checks à la Google or required a Microsoft account to login, they’d just shrug and go “create a Microsoft account then, stop bitching”. They don’t realise they are being boiled and don’t care. Consoomer behaviour.
Or we just realize that GitHub without logging in is a service we are getting for free. And when there’s something free, there’s someone trying to exploit it. Using GitHub while logged in is also free and has none of these limits, while allowing them to much easier block exploiters.
I would like to remind you that you are arguing for a monopolist. I’d agree with you if it were for a startup or mid-sized company that had lots of competition and was providing a good product being abused by competitors or users. But Github has a quasi-monopoly, is owned by a monopolist that is part of the reason other websites are being bombarded by requests (aka, they are part of the problem), and you are sitting here arguing that more people should join the monopoly because of an issue they created.
Can you see the flaws in reasoning in your statements?
No. I cannot find the flaws in my reasoning. Because you are not attacking my reasoning, you are saying that i am on the side of the bad people, and the bad people are bad, and you are opposed to the bad people, therefore you are right.
The world is more than black or white. GitHub rate-limiting non-logged-in users makes sense, and is the expected result in the age of web scrapping LLM training.
Yes, the parent company of GitHub also does web scrapped for the purpose of training LLMs. I don’t see what that has to do with defending themselves from other scrappers.
Company creates problem. Requires users to change because of created problem. You defend company creating problem.
That’s the logical flaw.
If you see no flaws in defending a monopolist, well, you cannot be helped then.
I don’t think Microsoft invented scrapping. Or LLM training.
Also, GitHub doesn’t have an issue with Microsoft scraping its data. They can just directly access whatever data they want. And rate-limiting non logged in accounts won’t affect Microsoft’s LLM training at all.
I’m not defending a monopolist because of monopolist actions. First of all because GitHub doesn’t have any kind of monopoly. There are plenty of git forges. And second of all. How does this make their position on the market stronger? If anything, it makes it weaker.
Open source repositories should rely on p2p. Torrenting repos is the way I think.
Not only for this. At any point m$ could take down your repo if they or their investors don’t like it.
I wonder if it would already exist and if it could work with git?
Git is p2p and distributed from day 1. Github is just a convenient website. If Microsoft takes down your repo, just upload to another system. Nothing but convenience will be lost.
Look into https://radicle.xyz/
Not entirely true. You lose tickets and PRs in that scenario.
I’ve heard git-bug a few times for decentralised issue tracking, never tried it but the idea is interesting
Yeah, pretty neat!
The project’s official repo should probably exist in a single location so that there is an authoritative version. At that point p2p is only necessary if traffic for the source code is getting too expensive for the project.
Personally I think the source hut model is closest to the ideal set up for OSS projects. Though I use Codeberg for my personal stuff because I’m cheap and lazy
I’m wary of external dependencies. They are cool now, but will they be cool in the future? Will they even exist?
One thing I think p2p excels is resiliance. People be still using eDonkey even if it’s abandoned.
A repo signature should deal with “fake copies”. It’s true we have the problem that BitTorrent protocol is not though for updating files, so a different protocol would be needed. I don’t even know how possible/practical it is. It’s true that any big project should probably host their own remote repo, and copy it on other platforms as needed. Github only repos was always a dangerous practice.
It’s true we have the problem that BitTorrent protocol is not though for updating files
Bittorrent v2 has updatable torrents
If you’re able to easily migrate issues etc to a new instance, then you don’t need to worry about a particular service providers getting shitty. At which point your main concern is temporary outages.
Perhaps this is more of a concern for some projects (e.g. anything that angers Nintendo’s lawyers). But for most, I imagine that the added complexity of distributed p2p hosting would outweigh the upsides.
Not saying it’s a bad idea, in fact I like it a lot, but I can see why it’s not a high priority for most OSS devs
I’ve been reading about it. But at some point I found that the parent organization run a crypto scam. Supposedly is not embedded into the protocol but they also said that the token is used to give rewards withing the protocol. That just made me wary of them.
Though the protocol did seen interesting. It’s MIT licensed I think so I suppose it could just be forked into something crypto free.
There’s nothing crypto in the radicle protocol. What I think you’re referring to are “drips” which uses crypto to fund opensource development (I know how terrible). It’s its own protocol built on top of ethereum and is not built into the radicle protocol.
This comes up every time someone mentions radicle and I think it happens because there’s a RAD crypto token and a radicle protocol. Beyond the similar names, it’s like mistaking bees for wasps because they look similar and not bothering to have a closer look.
Drips are funding the development of gitoxide, BTW, which is a Rust reimplementation of git. I wouldn’t start getting suspicious of gitoxide sneaking in a crypto protocol just because it’s funded by crypto. If we attacked everything funded by the things we consider evil, well everything opensource made by GAFAM would have to go: modern video streaming (HLS by Apple), Android (bought by Google), LSPs (popularised and developed by Microsoft), OBS (sponsored by Google through YouTube and by Amazon through Twitch), and much much more.
The thing is that the purpose of such a system is to run away from enshitificacion.
If they are so crypto adjacent is like a enshitificacion speedrun.
If I’m going to stay in a platform that just care for the money I might as well stay in corpo platforms. I’m not going to the trouble of changing platform and using new systems to keep getting being used so others can enrich.
Git itself doesn’t have crypto around it. This shouldn’t have either.
And this is not even against crypto as a concept, which is fine by me. It’s against using crypto as a scam to get a quick buck out of people who doesn’t know better.
If I’m going to stay in a platform that just care for the money
Where are you getting this information from? How is radicle just caring about money?
I’m not going to the trouble of changing platform and using new systems to keep getting being used so others can enrich.
Who is getting rich and how?
Answer to both questions is the crypto scheme they have created. There is no logical explanation to it. We have seen it happen countless times before.
They could ask for crypto donations and that would be totally fine. But they are building a crypto scheme. And crypto schemes are build as pyramid schemes to get money out of vulnerable people. Anyone who make such a thing is not trustable.
Who is building a cryptoscheme? Radicle developers aren’t building a cryptoscheme. Again, radicle is not crypto, it’s a decentralised git forge.
Torrenting doesn’t deal well with updating files.
And you have another problem: how do you handle bad actors spamming the download ?
That’s probably why github does that.That’s true. I didn’t think of that.
IPFS supposedly works fine with updating shares. But I don’t want to get closer to that project as they had fallen into cryptoscam territory.
I’m currently reading about “radicle” let’s see what the propose.
I don’t get the bad actors spamming the download. Like downloading too much? Torrent leechers?
EDIT: Just finished by search sbout radicle. They of course have relations with a cryptomscam. Obviously… ;_; why this keep happening?
There’s literally nothing about crypto in radicle from my reading, cryptography and crypto currency are not synonymous.
Ah because they also have a different project for a crypto payment platform for funding open source development.
Edit again: it seems pretty nifty actually, why do you think it’s a scam? Just because crypto?
I have a question: why do lemmy dev keep using microsoft github?
Yeah, shoulda use https://gitflic.ru/
Probably getting hammered by ai scrapers
you mean, doin’ what microsoft and their ai ‘partners’ do to others?
Yeah but they’re allowed to do it because they have brazillions of dollars.
They literally own GitHub. Brazillions well spent.
The funny thing is that rate limits won’t help them with genai scrapers
Everything seems to be. There was a period where you could kinda have a sane experience browsing over a VPN or otherwise using a cloud service IP range endpoint but especially the past 6 months or so things have gotten worse exponentially by the week. Everything is moving behind cloudflare or other systems
Its always blocked me from searching in firefox when I’m logged out for some reason.
Wow so surprising, never saw this coming, this is my surprised face. :-l
If Microsoft knows how to do one thing well, it’s killing a successful product.
I came here looking for this comment. They bought the service to destroy it. It’s kind of their thing.
Github has literally never been doing better. What are you talking about??
We are talking about EEE
What has Microsoft extinguished lately? I’m not a fan of Microsoft, but I think EEE is a silly thing to reference because it’s a strategy that worked for a little while in the 90s that Microsoft gave up on a long time ago because it doesn’t work anymore.
Like, what would be the purpose of them buying GitHub just to destroy it? And if that was their goal, why haven’t they done it already? Microsoft is interested in one thing: making money. They’ll do evil things to make money, just like any other big corporation, but they don’t do evil things just for the sake of being evil. It’s very much in their business interest to be seen as trustworthy, and being overly evil runs counter to that need.
It’s a slow process
RIP Skype
we could have had bob or clippy instead of ‘cortana’ or ‘copilot’
If Cortana was named Bob I don’t think people would have less of a problem with it
Microsoft really should have just leaned into it and named it Clippy again.
Crazy how many people think this is okay, yet left Reddit cause of their API shenanigans. GitHub is already halfway to requiring signing in to view anything like Twitter (X).
It’s not the same making API costs unbearable for a social media user and limiting the rate non-logged-in users.
You can still use GitHub without being logged in. You can still use GitHub without almost any limit on a free account.
You cannot even use reddit on a third party app with an account with reddit gold.
They make you sign in to use search, on code anyways.
Which i hate so much anytime i want to quickly look for something
