- cross-posted to:
- lemmydirectory@lemmy.dbzer0.com
- fuck_ai@lemmy.world
- cross-posted to:
- lemmydirectory@lemmy.dbzer0.com
- fuck_ai@lemmy.world
You must log in or register to comment.
I know she’s exaggerating but this post yet again underscores how nobody understands that it is training AI which is computationally expensive. Deployment of an AI model is a comparable power draw to running a high-end videogame. How can people hope to fight back against things they don’t understand?
It’s closer to running 8 high-end video games at once. Sure, from a scale perspective it’s further removed from training, but it’s still fairly expensive.
How exactly did you come across this “fact”?
I compared the TDP of an average high-end graphics card with the GPUs required to run big LLMs. Do you disagree?
I do, because they’re not at full load the entire time it’s in use
They are, it’d be uneconomical not to use them fully the whole time. Look up how batching works.
I mean I literally run a local LLM, while the model sits in memory it’s really not using up a crazy amount of resources, I should hook up something to actually measure exactly how much it’s pulling vs just looking at htop/atop and guesstimating based on load TBF.
Vs when I play a game and the fans start blaring and it heats up and you can clearly see the usage increasing across various metrics
deleted by creator
My guy, we’re not talking about just leaving a model loaded, we’re talking about actual usage in a cloud setting with far more GPUs and users involved.
He isn’t talking about locally, he is talking about what it takes for the AI providers to provide the AI.
To say “it takes more energy during training” entirely depends on the load put on the inference servers, and the size of the inference server farm.
really depends. You can locally host an LLM on a typical gaming computer.
Yeh but those local models are usually pretty underpowered compared to the ones that run via online services, and are still more demanding than any game.
You can, but that’s not the kind of LLM the meme is talking about. It’s about the big LLMs hosted by large companies.
True, and that’s how everyone who is able should use AI, but OpenAI’s models are in the trillion parameter range. That’s 2-3 orders of magnitude more than what you can reasonably run yourself
This is still orders of magnitude less than what it takes to run an EV, which are an eco-friendly form of carbrained transportation. Especially if you live in an area where the power source is renewable. On that note, it looks to me like AI is finally going to be the impetus to get the U.S. to invest in and switch to nuclear power – isn’t that altogether a good thing for the environment?
deleted by creator
Well that’s sort of half right. Yes you can run the smaller models locally, but usually it’s the bigger models that we want to use. It would also be very slow on a typical gaming computer and even a high end gaming computer. To make it go faster not only is the hardware used in datacenters more optimised for the task, it’s also a lot faster. This is both a speed increase per unit as well as more units being used than you would normally find in a gaming PC.
Now these things aren’t magic, the basic technology is the same, so where does the speed come from? The answer is raw power, these things run insane amounts of power through them, with specialised cooling systems to keep them cool. This comes at the cost of efficiency.
So whilst running a model is much cheaper compared to training a model, it is far from free. And whilst you can run a smaller model on your home PC, it isn’t directly comparable to how it’s used in the datacenter. So the use of AI is still very power hungry, even when not counting the training.
nice name btw
Not at all. Not even close.
Image generation is usually batched and takes seconds, so 700W (a single H100 SXM) for a few seconds for a batch of a few images to multiple users. Maybe more for the absolute biggest (but SFW, no porn) models.
LLM generation takes more VRAM, but is MUCH more compute-light. Typically one has banks of 8 GPUs in multiple servers serving many, many users at once. Even my lowly RTX 3090 can serve 8+ users in parallel with TabbyAPI (and modestly sized model) before becoming more compute bound.
So in a nutshell, imagegen (on an 80GB H100) is probably more like 1/4-1/8 of a video game at once (not 8 at once), and only for a few seconds.
Text generation is similarly efficient, if not more. Responses take longer (many seconds, except on special hardware like Cerebras CS-2s), but it parallelized over dozens of users per GPU.
This is excluding more specialized hardware like Google’s TPUs, Huawei NPUs, Cerebras CS-2s and so on. These are clocked far more efficiently than Nvidia/AMD GPUs.
…The worst are probably video generation models. These are extremely compute intense and take a long time (at the moment), so you are burning like a few minutes of gaming time per output.
ollama/sd-web-ui are terrible analogs for all this because they are single user, and relatively unoptimized.
How about, fuck AI, end story.
how about, fuck capitalism? Have you lost sight of the goal?
deleted by creator
All of them at their disposal, we should get rid of all tools
deleted by creator
So mad at a joke
deleted by creator
I did, as a matter of fact, fuck AI.
But then the rage machine couldn’t rage
there is so much rage today. why don’t we uh, destroy them with facts and logic
Hahaha at this point even facts and logic is a rage inducing argument. “My facts” vs “Your facts”
You thought blind anger came from well informed opinions?
She’s not exaggerating, if anything she’s undercounting the number of tits.
deleted by creator
I mean, continued use of AI encourages the training of new models. If nobody used the image generators, they wouldn’t keep trying to make better ones.
you are correct, and also not in any way disagreeing with me.
I try lol.
TBH most people still use old SDXL finetunes for porn, even with the availability of newer ones.
Right, but that’s kind of like saying “I don’t kill babies” while you use a product made from murdered baby souls. Yes you weren’t the one who did it, but your continued use of it caused the babies too be killed.
There is no ethical consumption under capitalism and all that, but I feel like here is a line were crossing. This fruit is hanging so low it’s brushing the grass.
“The plane is flying, anyway.”
Are you interpreting my statement as being in favour of training AIs?
I’m interpreting your statement as “the damage is done so we might as well use it”
And I’m saying that using it causes them to train more AIs, which causes more damage.I agree with your second statement. You have misunderstood me. I am not saying the damage is done so we might as well use it. I am saying people don’t understand that it is the training of AIs which is directly power-draining.
I don’t understand why you think that my observation people are ignorant about how AIs work is somehow an endorsement that we should use AIs.
I guess.
It still smells like an apologist argument to be like “yeah but using it doesn’t actually use a lot of power”.
I’m actually not really sure I believe that argument either, through. I’m pretty sure that inference is hella expensive. When people talk about training, they don’t talk about the cost to train on a single input, they talk about the cost for the entire training. So why are we talking about the cost to infer on a single input?
What’s the cost of running training, per hour? What’s the cost of inference, per hour, on a similarly sized inference farm, running at maximum capacity?Maybe you should stop smelling text and try reading it instead. :P
Running an LLM in deployment can be done locally on one’s machine, on a single GPU, and in this case is like playing a video game for under a minute. OpenAI models are larger than by a factor of 10 or more, so it’s maybe like playing a video game for 15 minutes (obviously varies based on the response to the query.)
It makes sense to measure deployment usage marginally based on its queries for the same reason it makes sense to measure the environmental impact of a car in terms of hours or miles driven. There’s no natural way to do this for training though. You could divide training by the number of queries, to amortize it across its actual usage, which would make it seem significantly cheaper, but it comes with the unintuitive property that this amortization weight goes down as more queries are made, so it’s unclear exactly how much of the cost of training should be assigned to a given query. It might make more sense to talk in terms of expected number of total queries during the lifetime deployment of a model.
You’re way overcomplicating how it could be done. The argument is that training takes more energy:
Typically if you have a single cost associated with a service, then you amortize that cost over the life of the service: so you take the total energy consumption of training and divide it by the total number of user-hours spent doing inference, and compare that to the cost of a single user running inference for an hour (which they can estimate by the number of user-hours in an hour divided by their global inference energy consumption for that hour).
If these are “apples to orange” comparisons, then why do people defending AI usage (and you) keep making the comparison?
But even if it was true that training is significantly more expensive that inference, or that they’re inherently incomparable, that doesn’t actually change the underlying observation that inference is still quite energy intensive, and the implicit value statement that the energy spent isn’t worth the affect on society
I had my energy company remove their LVTC smart meter this week after they started using it to shut off our condenser unit during our 100 degree days
The fact that it exists at all is bad enough, but they were doing this at a time when our AC was already malfunctioning due to low refrigerant. On the day they first shut it off, our house reached 94 degrees.
The program that the previous owner signed up for that enabled them to do this gave them a fucking two dollar a month discount.
I use a smart thermostat to optimize my home conditioning - having a second meter fucking with my schedule ends up making us all miserable. Energy providers need to stop fucking around and just build out their infrastructure to handle worst case peak loads, and enable customers to install solar to reduce peak loading to begin with.
The other thing that kills me about this is that our provider administers our city’s solar electric subsidy program themselves. When i had them come out to give us a quote, they inflated their price by more than 100% because they knew what our electricity bill was. All they did was take our average monthly bill and multiplied it by the repayment period. I could have been providing them more energy to the grid at their peak load if they hadn’t tried scamming me.
FUCK private energy providers.
our city’s solar electric subsidy program
It sounds like there’s two different things there. There’s a solar installation (hardware, etc.), and there’s likely some kind of net metering program (where they pay you or give you credit for electricity you generate). That paragraph sounds like the first, but the phrase sounds like the second.
You shouldn’t have to go through them for the solar installation, if your conditions accommodate it. Granted, the conditions don’t apply to everyone. You’ll want to have a suitable roof that ideally faces south-ish, own your home, and plan to stay there for at least 10 years. In the US, you also kind of need to get it done within this calendar year, which is a rough ask, before the federal 30% tax credit goes away. But maybe you can find an installer that isn’t trying to scam you quite as much.
(It’s early and cloudy today.)
Your HA dashboard derailed this conversation for me. lol.
I would love to know more about the equipment you are using to push this info into your HA.
I’ll write up a post now in !solarpunk@slrpnk.net
Sorry, maybe I wasn’t being clear.
My area has solar incentive programs that are run through the energy utility - meaning the state makes available zero-interest loans for the purposes of solar installation, but those loans are only available through an entity partnered with our utility. They limit the number of homes in each area that are eligible through this program so that solar generation never exceeds demand. Our home was eligible through the program, so I had them come out to give us a quote. Our utility is also transitioning to surge pricing and smart metering, so there’s a pretty high demand for solar installation in my area and they know that they’d lose out on a lot of revenue if everyone installed their own solar systems.
A part of that process was them asking for the last year of energy bills, along with taking measurements and doing daylighting analysis on our roof area. At the end, they gave us a quote for a 15 year loan for the equipment and installation, and it just so happened that the monthly payment was the same as our average energy bill. I work in AEC and I know what solar panels cost, and they had inflated their price by more than double what it would cost at market rate.
Of course I could install my own panels, but it would be out-of-pocket and I would have to seek out and apply for out-of-state incentive programs myself, but I can’t afford the up-front costs and the loan terms don’t make sense for how long we’ll be in this house. Id love nothing more than to do it myself, even at a loss if that’s what it took, but I have a spouse that is less spiteful than I am.
more than double what it would cost at market rate
I definitely paid more for labor than for materials. My payoff time is about 13 years with a Tesla Powerwall 3, maybe a bit less now that I have an EV. I had a team of 4 guys plus an electrician here for about five days.
I did go with a slightly more reputable company that charged slightly more, but I would have gone elsewhere if it was a huge difference.
Maybe I should get around to making a post in !Solarpunk@slrpnk.net or something, even though it isn’t very punk.
I’m factoring in labor. It was an extremely bad deal - they were praying on the fact most home owners do not have familiarity with solar installation pricing.
Like I said, I would love to still do it on my own, but it just doesn’t make sense for our household.
Peak load of households is not during peak solar power generation. Households installing pv isn’t a solution to what you described.
Today, you could also use a battery to buy power during mid day and use it in the evening when you need it the most.
In moderate climates in the US, peak loads are typically the hottest and sunniest hours of the day since condenser units are the most energy-hungry appliance in most homes. Clouds notwithstanding, peak solar generation would typically align (or closely align) with peak load time.
Batteries would also help a lot - they should definitely be subsidizing the installation of those as well but unfortunately they aren’t yet (at least not in my state).
This is incorrect. Look up the “duck curve” or if you prefer real-world examples look at the California electricity market (CAISO) where they have an excellent “net demand curve” that illustrates the problem.
This curve has changed somewhat since this study in 2016. More efficient home insulation, remote working, and energy-efficient cooling systems have large impact in this pattern. But assuming you have a well-insulated home, setting your thermostat to maintain a consistent temperature throughout the day will shift this peak earlier and lower the peak load at sunset, when many people are returning home. More efficient heat pumps with variable pressure capabilities also helps this a lot, too.
Given just how many variables are involved, it’s better to assume peak cooling load to be mid-day and work toward equalizing that curve, rather than reacting to transient patterns that are subject to changes in customer behavior. Solar installations are just one aspect of this mitigation strategy, along with energy storage, energy-efficient cooling systems, and more efficient insulation and solar heat gain mitigation strategies.
If we’re discussing infrastructure improvements we might as well discuss home efficiency improvements as well.
Study or no study, you can see this problem in the real world https://www.caiso.com/todays-outlook#section-net-demand-trend
Ok now go just one step further and ask yourself what variables factor into this.
There’s a reason that pattern exists, and it isn’t because solar and cooling hours don’t align.
the difference between demand and net demand in that graph is purely solar/wind generation, isn’t it?
Do you have a source for the cooling off effect of the duck curve?
Following is a 2 year old article hinting an increase in the effect https://www.powermag.com/epri-head-duck-curve-now-looks-like-a-canyon/ afaik it hasn’t changed much but I’m open to news
I’m not really saying that the curve itself is changing (sorry, I was really not clear), only that those other variables reduce actual energy demand later in the day because of the efficiency gains and thermal banking that happens during the peak energy production. The overproduction during max solar hours is still a problem. Even if the utility doesn’t have a way of banking the extra supply, individual customers can do it themselves at a smaller scale, even if just by over-cooling their homes to reduce their demand after sundown.
Overall, the problem of the duck curve isn’t as much about maxing out the grid, it’s about the utility not having instantaneous power availability when the sun suddenly goes down. For people like me who work from home and have the flexibility to keep my home cool enough to need less cooling in the evening, having solar power means I can take advantage of that free energy and bank it to reduce my demand in the evening.
I get what you were saying now, but having solar would absolutely reduce my demand during peak hours.
It’s a neat idea to over-cool in order to reduce consumption later on!
I watch big state and national grid loads (for fun) and I see two distinct peaks: 7-8AM when everyone goes to work, and then around 5-7 PM when people commute home and heat up dinner.
Otherwise it’s a linear diagonal curve coinciding with temperatures.
I personally try to keep my own energy usage a completely flat line so I can benefit from baseline load generator plants like nuclear (located not that far away).
I personally try to keep my own energy usage a completely flat line so I can benefit from baseline load generator plants like nuclear (located not that far away).
If you consume energy during peak hours you are a peak load consumer. Consuming in other hours doesn’t change this fact.
Your personal energy use pattern does not determine where you are drawing your power from, wtf logic is this?
Why do you want a subsidy for batteries? Installing batteries at a large scale at homes is incredibly expensive compared to an off site battery. Especially with regards to the move towards hydrogen.
For the same reason we want to subsidize solar production in residential construction even though it’s more efficient and cost-productive to do it at-scale. Having energy production and storage at the point of use reduces strain on power infrastructure and helps alleviate the types of load surging ayyy is talking about.
It’s not a replacement for modernizing our power grids, too - it simply helps to make them more resilient.
That’s understandable but do we need it now? Neither pv nor batteries last forever. I’m just not sure if we need them now (or short-medium term future). But I’m not in the position to decide upon it
I don’t know how to answer that question.
Can also just overcool if well insulated.
Smart meters with this ability are great, when done well. Without them they have the ability to turn off all of your power if they need to. If they can’t keep up with demand, they have to turn things off. It’s better for them to have the ability to shut off a few appliances or decrease your AC usage rather than shut people down entirely.
People always complain that they don’t want to give the energy company power over their electricity, but they already do. However, without this their power is total, and only total. With it they can moderate it. It’s better if everyone has a smart meter instead of only people who care about others, and greedy people only look out for themselves.
I agree though, fuck private providers.
I agree though, fuck private providers
If it was a public utility i’d feel very differently.
deleted by creator
I’m here for the pic
we had three tiddied aliens in total recall, like 40 years ago. we don’t need AI to give us more tits.
I should have figured the Rick and Morty episode was a reference to something.
Makes me think about South Park and watching it as it aired when I was a kid. There were so many things I missed because I hadn’t seen any of the source material for a lot of the jokes.
Watching it all again 25 years later and damn, even better the second time around when you’ve seen all the shit they’re parodying.
I didn’t know Rick and Morty referenced this but I shouldn’t be surprised. Same here with the missed references. It makes rewatching older content not just nostalgic but also a fun discovery.
we don’t need … more tits.
Blasphemy!!
deleted by creator
I feel like i’ve read a very similar argument somewhere recently, but i have difficulty remembering it precisely. It went something like this:
- If a company kills 5 people, it was either an accident, an unfortunate mishap, a necessity of war (in case of the weapons industry) or some other bullshit excuse.
- If the people threaten to kill 5 billionaires, they’re charged with “terrorism” (see Luigi Mangione’s case).
deleted by creator
5 tits, that’s awesome.
Does an image like that exist?! Asking for a friend.
Two wrongs don’t make a right. And if your neighbour is dosing the neighbourhood with gasoline while wildfires are on the horizon, you smack him, you don’t go and get your own can.
Switch all traditional AC to being powered by Heat Pumps, destroy all private jets, ban recreational flights and power AI responsibly or not at all.
AC systems are heat pumps so I’m not sure what you mean there.
Most domestic flights should be replaced by highspeed rail anyways.
Agreed
Switch all traditional AC to being powered by Heat Pumps
Aren’t traditional AC’s (assuming you are somewhere where AC is a thing used to cool the air) already heat pumps?
In my attempt at brevity, I articulated myself wrong, totally my bad. I would like the old school systems replaced with either air source heat pumps or ground source heat pumps, backed up with on-site solar and batteries. Modern heat pump systems can heat and cool and are much more efficient than AC as generally installed.
The efficiency gains from an air source heat pumps are on the heating cycle, not the cooling cycle, since you are moving heat around instead of having to generate heat via combustion or big heating elements. When acting as an air conditioner, the efficiency is the same.
Ground source heat pumps (GSHPs) are generally the most efficient, achieving 350-500% efficiency by leveraging stable underground temperatures, though they have higher installation costs. Air source heat pumps (ASHPs) are also highly efficient at 250-400%, extracting heat from the air, but their performance can be affected by extreme outdoor temperatures. In contrast, a traditional gas boiler for heating is around 90-95% efficient, while separate air conditioning units cool, but neither offers the combined, high-efficiency performance of a heat pump. Therefore, for overall energy savings and reduced environmental impact, heat pumps are the superior choice for both heating and cooling.
An AC is an “air heat pump”. The only difference between an AC and what we call a “heat pump” is a reversing valve, which can send refrigerant the other way to heat the interior instead of cooling it.
They’re literally the same thing.
That’s the only definitional difference. In practice there are other differences. My modern cold climate heat pump has a variable speed compressor whereas my previous traditional AC did not. The variable speed allows the system to ramp up and down on both heating and cooling, letting the system run all the time even at a very low level when the demand is low.
The traditional AC’s single speed compressor ended up doing a lot of short cycling when cooling demand was low and shutting down completely when cooling demand was too high (to prevent overheating and compressor damage). The variable speed compressor of the modern heat pump is designed for continuous operation over many hours, even when the temperature outside is extremely high, without overheating. I believe it’s able to back off the compressor speed when the cooling demand exceeds capacity though I have yet to see the system be unable to keep up, despite the unit itself being a lot smaller than the old AC.
I genuinely think the oversimplification of what a heat pump is and how it compares to AC is malignant. It’s like comparing a rickshaw to a bullet train.
They’re literally the same thing.
A traditional air conditioner provides only cooling by moving heat out of your home, primarily contributing to summer electricity peaks. In contrast, a heat pump offers both heating and cooling by simply reversing the refrigerant flow, making it a more versatile and energy-efficient solution for year-round comfort. While heat pumps increase overall electricity demand by electrifying heating, they also shift energy consumption patterns, creating a new winter peak for the grid to manage. However, this increased electrical load presents an opportunity for demand response, allowing smart heat pumps to adjust usage during peak times to balance the grid. Ultimately, widespread heat pump adoption, powered by a decarbonising electricity supply, is crucial for reducing fossil fuel reliance and achieving a greener energy system, albeit requiring significant grid infrastructure upgrades.
Is this an AI-generated response? On a post demonizing AI? Lol bold
Agreed, but this is like comparing your neighbor burning 1 million acres to you having a bonfire. The scale is the problem. We should absolutely take individual responsibility; however, our small impact is only felt when we band together.
our small impact is only felt when we band together
It is also offset immediately when unregulated corporations use the saved energy to sell us the next dumb thing.
Great, now this might work with my neighbor, but how exactly do I smack mega corps and the state? Are we talking eco terrorism here or do you have some other idea that hasn’t been tried in the last decades?
I mean, climate change isn’t new but humanity still fucks up the planet and that does not seem to change. Why should we have to sweat at home while professionalized greed burns down everything around us? I will gladly take individual responsibility, but not alone.
Actually, a failing power grid here and there might act as a wake-up call and then we can start talking about solutions, not just symptomatic treatment.
Talking about direct action or even a mildly disruptive protest will probably get you moderated here, and in trouble in real life. It feels like the only options “allowed” are stern words. At least a progressive like Zohran won the primary in NYC, but we’ll need a lot more of that to make a difference.
On the other hand, Luigi is considered by many a hero.
Meanwhile I’m down town I’m my city cleaning windows in office buildings that are 75% empty but the heat or ac is blasting on completely empty floors and most of the lights are on.
The HVAC does serve a purpose, it reduces the moisture in the building, which would otherwise ruin the building
Apparently those flex requests aren’t about saving the environment; it mostly has something to do with power grids needing to operate at a certain range at all times and spikes in demand fucks with that. That is to say, if they increase the energy produced, they actually have other problems at times of reduced demand, like at night or winter time. I’m paraphrasing from a pretty brief thing I heard over the radio, though, so I’m sure it’s more complicated.
They still cut corners and cause wild fires because they don’t update their tech and prefer to have things fail, though. Plenty of reasons to hate your private utility companies, lol
I have a crazy theory that requests like these will actually push people to care more about and take action on global warming.
AI without demand but also destroying the planet. Typical complex line of thought on the Internet.
79 is like my ideal temp. Cities must love me.
deleted by creator
I continuously have to remind myself that Fahrenheit is a thing so I don’t get a panic attack when I read comments like yours.
Laughs in total recall
