• 3 Posts
  • 15 Comments
Joined 2 years ago
cake
Cake day: June 14th, 2023

help-circle
  • suy@programming.devtoFuck AI@lemmy.worldThe Perfect Response
    link
    fedilink
    arrow-up
    3
    arrow-down
    4
    ·
    3 months ago

    I don’t know where you got that image from. AllenAI has many models, and the ones I’m looking at are not using those datasets at all.

    Anyway, your comments are quite telling.

    First, you pasted an image without alternative text, which it’s harmful for accessibility (a topic in which this kind of models can help, BTW, and it’s one of the obvious no-brainer uses in which they help society).

    Second, you think that you need consent for using works in the public domain. You are presenting the most dystopic view of copyright that I can think of.

    Even with copyright in full force, there is fair use. I don’t need your consent to feed your comment into a text to speech model, an automated translator, a spam classifier, or one of the many models that exist and that serve a legitimate purpose. The very image that you posted has very likely been fed into a classifier to discard that it’s CSAM.

    And third, the fact that you think that a simple deep learning model can do so much is, ironically, something that you share with the AI bros that think the shit that OpenAI is cooking will do so much. It won’t. The legitimate uses of this stuff, so far, are relevant, but quite less impactful than what you claimed. The “all you need is scale” people are scammers, and deserve all the hate and regulation, but you can’t get past those and see that the good stuff exists, and doesn’t get the press it deserves.


  • suy@programming.devtoFuck AI@lemmy.worldThe Perfect Response
    link
    fedilink
    arrow-up
    13
    arrow-down
    9
    ·
    3 months ago

    Which ones? Name one.

    What’s wrong with what Pleias or AllenAI are doing? Those are using only data on the public domain or suitably licensed, and are not burning tons of watts on the process. They release everything as open source. For real. Public everything. Not the shit that Meta is doing, or the weights-only DeepSeek.

    It’s incredible seeing this shit over and over, specially in a place like Lemmy, where the people are supposed to be thinking outside the box, and being used to stuff which is less mainstream, like Linux, or, well, the fucking fediverse.

    Imagine people saying “yeah, fuck operating systems and software” because their only experience has been Microsoft Windows. Yes, those companies/NGOs are not making the rounds on the news much, but they exist, the same way that Linux existed 20 years ago, and it was our daily driver.

    Do I hate OpenAI? Heck, yeah, of course I do. And the other big companies that are doing horrible things with AI. But I don’t hate all in AI because I happen to not be an ignorant that sees only the 99% of it.


  • I’ve made several Qt apps (in C++) easily packaged using AppImage. Perhaps OBS is harder because they require some level of integration with the hardware (e.g. the virtual camera perhaps requires something WRT drivers, I don’t know), but in the general case of a Qt app doing “normal GUI stuff” and “normal user stuff” is a piece of cake. To overcome the glibc problem, it’s true that it’s recommended using an old distro, but it’s not a must. Depends on what you want to support.

    As a user, I prefer a native package, though (deb in my case).





  • OpenAI doesn’t produce LLMs only. People are gonna be paying for stuff like Sora or DallE. And people are also paying for LLMs (e.g. Copilot, or whatever advanced stuff OpenAI offers in their paid plan).

    How many, and how much? I don’t know, and I am not sure it can ever be profitable, but just reducing it to “chains of bullshit” to justify that it has no value to the masses seems insincere to me. ChatGPT gained a lot of users in record time, and we know is used a lot (often more than it should, of course). Someone is clearly seeing value in it, and it doesn’t matter if you and I disagree with them on that value.

    I still facepalm when I see so many people paying for fucking Twitter blue, but the fact is that they are paying.



  • Lol. We’re as far away from getting to AGI as we were before the whole LLM craze. It’s just glorified statistical text prediction, no matter how much data you throw at it, it will still just guess what’s the next most likely letter/token based on what’s before it, that can’t even get it’s facts straith without bullshitting.

    This is correct, and I don’t think many serious people disagree with it.

    If we ever get it, it won’t be through LLMs.

    Well… depends. LLMs alone, no, but the researchers who are working on solving the ARC AGI challenge, are using LLMs as a basis. The one which won this year is open source (all are if are eligible for winning the prize, and they need to run on the private data set), and was based on Mixtral. The “trick” is that they do more than that. All the attempts do extra compute at test time, so they can try to go beyond what their training data allows them to do “fine”. The key for generality is trying to learn after you’ve been trained, to try to solve something that you’ve not been prepared for.

    Even OpenAI’s O1 and O3 do that, and so does the one that Google has released recently. They are still using heavily an LLM, but they do more.

    I hope someone will finally mathematically prove that it’s impossible with current algorithms, so we can finally be done with this bullshiting.

    I’m not sure if it’s already proven or provable, but I think this is generally agreed. just deep learning will be able to fit a very complex curve/manifold/etc, but nothing more. It can’t go beyond what was trained on. But the approaches for generalizing all seem to do more than that, doing search, or program synthesis, or whatever.



  • Now I sail the high seas myself, but I don’t think Paramount Studios would buy anyone’s defence they were only pirating their movies so they can learn the general content so they can produce their own knockoff.

    We don’t know exactly how they source their data (and that is definitely shady), but if I can gain access to a movie in a legal way, I don’t see why I would not be able to gather statistics from said movie, including running a speech to text model to caption it, then make statistics of how many times a few words were used, and followed by which ones. This is an oversimplified explanation of what a LLM does, but it’s the fairest I can come up, and it would be legal to do so. The models are always orders of magnitude smaller than the data they are trained on.

    That said, I don’t imply that I’m happy with the state of high tech companies, the AI hype, the energy consumption, or the impact on the humble people. But I’ve put a lot of thought into this (and learning about machine learning for real), and I think this is not a ML problem, but a problem in the economic, legal and political system. AI hype is just a symptom.



  • “Theft” is never a technically accurate word when dealing with the so called “intellectual property”, because the digital content being copied without authorization is legal in tons of cases, and because, come on, property is very explicitly exclusive. I cannot copy my house or my car, but I can make copies of my works for virtually 0 cost.

    Using data for training ML models is even explicitly allowed in some jurisdictions (e.g. Japan), and is likely to be fair use everywhere else. LLMs are very transformative, and while they often can produce verbatim copies of fragments of copyrighted works, they don’t store the whole works or significant pieces of them.

    Don’t get me wrong, I don’t like big companies making big money. I would not mind a law that would force models to be open sourced. But restricting them to train their models on public data by restricting fair use, it would harm them very little (they could pay something if they are making some profit), while small researchers or companies would never be able to compete, because they would not have the upfront costs, nor the economic engineering to disguise profits and pay less.


  • this has to do with writing ‘better’ code, which has proved impossible over and over again

    I can’t speak for C, as I don’t follow it that much, but for C++, this is just not fair. It has been proven repeatedly that it can be done better, and much better. Each iteration has made so many things simpler, more productive, and also safer. Now, there are two problems with what I just said:

    • That it has been done safer, that doesn’t mean that everyone makes good use of it.
    • That it has been done safer, doesn’t mean that everything is fixable, and that it’s on the same level of other, newer languages.

    If that last part is what you mean, fine. But the way that you phrased (and that I quoted) is just not right.

    At this point it’s literally easier to slowly port to a better language than it is to try and ‘fix’ C/C++.

    Surely not for everything. Of course I see great value if I can stop depending on OpenSSL, and move to a better library written in a better language. Seriously looking forward for the day when I see dynamic libraries written in Rust in my package manager. But I’d like to see what’s the plan for moving a large stack of C and C++ code, like a Linux distribution, to some “better language”. I work everyday on such a stack (e.g. KDE Neon in my case, but applicable to any other typical distro with KDE or GNOME), and deploy to customers on such a stack (on Linux embedded like Yocto). Will the D-Bus daemon be written in Rust? Perhaps. Systemd? Maybe. NetworkManager, Udisks, etc.? Who knows. All the plethora of C and C++ applications that we use everyday? Doubtful.


  • I’m not fully sure what the intent of the joke is, but note that yes, it’s true that a header typically just has the prototype. However, tons of more advanced libraries are “header-only”. Everything is in a single header originally, in development, or it’s a collection of headers (that optionally gets “amalgamated” as a single header). This is sometimes done intentionally to simplify integration of the library (“just copy this files to your repo, or add it as a submodule”), but sometimes it’s entirely necessary because the code is just template code that needs to be in a header.

    C++ 20 adds modules, and the situation is a bit more involved, but I’m not confident enough of elaborating on this. :) Compile times are much better, but it’s something that the build system and the compilers needs to support.