• MystikIncarnate@lemmy.ca
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    2 hours ago

    IT guy checking in.

    The only time I’ve even seen drive temp sensor alarms is on server raid arrays and other similar hard drives/SSDs… Never in my life have I seen one available on a consumer device, nor have I seen any alarm for and drive temp, go off. It just doesn’t happen.

    IMO, this is one of those language barriers where people call their computer chassis (and everything in it) the “hard drive”.

    Applying that assumption, their updated statement is: His computer over heated.

    Idk what kind of shit system he’s running on that 60k rows would cause overheating, but ok.

  • rumba@lemmy.zip
    link
    fedilink
    English
    arrow-up
    87
    ·
    1 day ago

    Unless I’m misreading it which is possible it’s awfully late, he said he processed 60,000 rows didn’t find what he was looking for but his hard drive overheated on the full pass.

    Discs don’t overheat because there was load. Even if he f***** up and didn’t index the data correctly (I assume it’s a relational database since he’s talking about rows) The disc isn’t just going to overheat because the job is big. It’s going to be lack of air flow or lack of heatsink.

    I guarantee you he was running on an external NVMe, and one of those little shitty-ass Chinese enclosures. Or maybe one of those self immolating SanDisk enclosures. Hell, maybe he’s on a desktop and he slept a raw NVMe on his motherboard without a heatsink

    There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.

  • nonentity@sh.itjust.works
    link
    fedilink
    arrow-up
    35
    ·
    1 day ago

    Either she knows something novel, where processing data using voice coils is somehow beneficial, or is someone who calls their computer a ‘hard drive’, which summarily negates any legitimacy of technical competence.

  • RussianBot8453@lemmy.world
    link
    fedilink
    arrow-up
    71
    ·
    2 days ago

    I’m a data engineer that processes 2 billion row 3000 column datasets every day, and I open shit in Excel with more than 60k rows. What the hell is this chick talking about?

    • zenpocalypse@lemm.ee
      link
      fedilink
      English
      arrow-up
      21
      ·
      2 days ago

      Seems like a good excuse to someone who doesn’t know what they’re doing and needs an excuse because why they haven’t completed it yet?

      The whole post is complete bs in multiple ways. So weird.

      • psivchaz@reddthat.com
        link
        fedilink
        arrow-up
        3
        ·
        1 day ago

        It sounds like Hollywood tech lingo. Like when you’re watching a movie or a TV show and the designated techy character starts just saying computer words that make no actual sense in the real world, but I guess in CSI: Idiottown the hard drives have severe overheating issues.

      • Akasazh@feddit.nl
        link
        fedilink
        arrow-up
        5
        ·
        2 days ago

        If you work for a boss that fundamentally misunderstands what you are doing, then misleading them into thinking you’re ‘hard at work, making decisions with consequences’ is the theatre you put up to keep the cash flowing.

        It’s one of the fundamental flows of autocracy, people try and represent what you want them to

    • person420@lemmynsfw.com
      link
      fedilink
      arrow-up
      17
      ·
      2 days ago

      Some interesting facts about excel I learned the hard way.

      1. It only supports about a million or so rows
      2. It completely screws up numbers if the column is a number and the number is over 15 digits long.

      Not really related to what you said, but I’m still sore about the bad data import that caused me days of work to clean up.

      • driving_crooner@lemmy.eco.br
        link
        fedilink
        arrow-up
        7
        ·
        edit-2
        1 day ago

        It completely screws up numbers if the column is a number and the number is over 15 digits long.

        I work in insurance in Brazil, by standards of our regulatory body, claims numbers must be a string of 20 numbers (zfill(20) if needed). You can’t imagine the amount of times excel had fucked me up rounding down the claim numbers, this is one of the first things I teach to my interns and juniors when they’re working with the claims databases.

      • Mniot@programming.dev
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 days ago

        The row limitation seems, to me, like an actually-good thing. Excel is for data where you might conceivably scroll up and down looking at it and 1M is definitely beyond the ability of a human even to just skim looking for something different.

        An older version of Excel could only handle 64k rows and I had a client who wanted large amounts of data in Excel format. “Oh sorry, it’s a Microsoft limitation,” I was thrilled to say. “I have no choice but to give you a useful summarization of the data instead of 800k rows (each 1000 columns wide) of raw data.”

        • frezik@midwest.social
          link
          fedilink
          arrow-up
          3
          ·
          edit-2
          1 day ago

          Some time ago, I heard a story of CS and Econ professors having lunch together. The Econ professor was excited that Excel was going to release a version that blew out the 64k row limit. The CS professor nearly choked on his lunch.

          Dependence on Excel has definitely caused bad papers to be published in the Econ space, and has had real world consequences. There was a paper years ago that stated that once a country’s debt gets above 120% of GDP, its economy goes into a death spiral. It was passed around as established fact by the sorts of politicians who justify austerity. Problem was, nobody could reproduce the results. Then an Econ undergrad asked the original author for their Excel spreadsheet, and they found a coding error in the formulas. Once corrected, the conclusion disappeared.

  • zalgotext@sh.itjust.works
    link
    fedilink
    arrow-up
    77
    arrow-down
    1
    ·
    edit-2
    18 hours ago

    my hard drive overheated

    So, this means they either have a local copy on disk of whatever database they’re querying, or they’re dumping a remote db to disk at some point before/during/after their query, right?

    Either way, I have just one question - why?

    Edit: found the thread with a more in-depth explanation elsewhere in the thread: https://xcancel.com/DataRepublican/status/1900593377370087648#m

    So yeah, she’s apparently toting around an external hard drive with a copy of the “multiple terabytes” large US spending database, running queries against it, then dumping the 60k-row result set to CSV for further processing.

    I’m still confused at what point the external drive overheats, even if she is doing all this in a “hot humid” hotel room that she can’t run any fans I guess because her kids were asleep?

    But like, all of that just adds more questions, and doesn’t really answer the first one - why?

      • spooky2092@lemmy.blahaj.zone
        link
        fedilink
        English
        arrow-up
        43
        ·
        2 days ago

        Plus, 60k is nothing. One of our customers had a database that was over 3M records before it got some maintenance. No issue with overheating lol

        • surph_ninja@lemmy.world
          link
          fedilink
          arrow-up
          23
          ·
          edit-2
          2 days ago

          I run queries throughout the day that can return 8 million+ rows easily. Granted, it takes few minutes to run, but it has never caused a single issue with overheating even on slim pc’s.

          This makes no fucking sense. 60k rows would return in a flash even on shitty hardware. And if it taxes anything, it’s gonna be the ram or cpu- not the hard drive.

        • AThing4String@sh.itjust.works
          link
          fedilink
          arrow-up
          12
          ·
          2 days ago

          I literally work with ~750,000 line exports on the daily on my little Lenovo workbook. It gets a little cranky, especially if I have a few of those big ones open, but I have yet to witness my hard drive melting down over it. I’m not doing anything special, and I have the exact same business-economy tier setup 95% of our business uses. While I’m doing this, that little champion is also driving 4 large monitors because I’m actual scum like that. Still no hardware meltdowns after 3 years, but I’ll admit the cat likes how warm it gets.

          750k lines is just for the branch specific item preferences table for one of our smaller business streams, too - FORGET what our sales record tables would look like, let alone the whole database! And when we’re talking about the entirety of the social security database, which should contain at least one line each in a table somewhere for most of the hundreds of millions of people currently living in the US, PLUS any historical records for dead people??

          Your hard drive melting after 60k lines, plus the attitude that 60k lines is a lot for a major database, speaks to GLARING IT incompetence.

        • Korhaka@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          2 days ago

          Pretty sure I run updates or inserts that count over 60k fairly often. No overheats. Select queries sometimes way higher.

      • Dave.@aussie.zone
        link
        fedilink
        arrow-up
        11
        arrow-down
        1
        ·
        2 days ago

        You’ve got it all wrong, in traditional computer terminology the “hard drive” is the box that sits under the desk that collects cat fluff and cigarette tar.

        /s …?

      • Fuck spez@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        5
        ·
        2 days ago

        I don’t think I’ve seen a brand new computer in the past decade that even had a mechanical hard drive at all unless it was purpose-built for storing multiple terabytes, and 60K rows wouldn’t even take multiple gigabytes.

    • zenpocalypse@lemm.ee
      link
      fedilink
      English
      arrow-up
      17
      ·
      edit-2
      2 days ago

      Even if it was local, a raspberry pi can handle a query that size.

      Edit - honestly, it reeks of a knowledge level that calls the entire PC a “hard drive”.

    • GoodEye8@lemm.ee
      link
      fedilink
      English
      arrow-up
      15
      ·
      2 days ago

      My one question would be “How?”

      What the hell are you doing that your hard drives are overheating? How do you even know it’s overheating as I’m like 90% certain hard drives (except NVMe if we’re being liberal with the meaning of hard drive) don’t even have temperature sensors?

      The only conclusion I can come to is that everything he’s saying is just bullshit.

      • Auli@lemmy.ca
        link
        fedilink
        English
        arrow-up
        12
        ·
        2 days ago

        They have temp sensors. But have never heard of a overheating drive.

        • xthexder@l.sw0.com
          link
          fedilink
          arrow-up
          7
          ·
          edit-2
          2 days ago

          Imo if they can’t max out their harddrive for at least 24 hours without it breaking, their computer was already broken. They just didn’t know it yet.

          Any reasonable SSD would just throttle if it was getting too hot, and I’ve never heard of a HDD overheating on its own, only if there’s some external heat sources, like running it in a 60°C room

          • Mniot@programming.dev
            link
            fedilink
            English
            arrow-up
            3
            ·
            2 days ago

            Can we think of any device someone might have that would struggle with 60k? Certainly an ESP32 chip could handle it fine, so most IoT devices would work…

            • zenpocalypse@lemm.ee
              link
              fedilink
              English
              arrow-up
              4
              ·
              2 days ago

              Right? There’s no part of that xeet that makes any real sense coming from a “data engineer.”

              Terrifying, really.

    • Adalast@lemmy.world
      link
      fedilink
      arrow-up
      6
      ·
      2 days ago

      Why? Because they feel the need to have local copies of sensitive financial information because… You know… They are computer security experts.

  • Onno (VK6FLAB)@lemmy.radio
    link
    fedilink
    arrow-up
    202
    ·
    2 days ago

    Wow.

    I’ve been processing a couple of billion rows of data on my machine, the fans didn’t even come on. WTF are they teaching “experts” these days, or has Elmo only hired people who claim that they can “wrangle data” and say “yes” ?

    • bleistift2@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      113
      ·
      2 days ago

      Even if querying data was processing-heavy and even if somehow the ‘hard drive’ got warm during this, then there still would need to be a hardware defect in order for the drive to overheat.

      • IrateAnteater@sh.itjust.works
        link
        fedilink
        arrow-up
        56
        ·
        2 days ago

        Yes, but this may be a symptom of an issue I’ve been seeing with younger programmers; they’ve siloed themselves so specifically into whatever programming they “specialize” in, that they become absolutely useless at dealing with absolutely anything else related to their job. And exasperating this issue is the fact that they’ve grown up with systems that “just work”. Windows, iOS, and android are all at the point where fucking around with hardware issues is very uncommon for the average person.

        Asking this guy to solve a hardware problem is like asking hime to tune a carburetor. He likely has not the slightest clue how to start.

        • bleistift2@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          7
          arrow-down
          8
          ·
          2 days ago

          That’s the price of specialization. Don’t ask a software engineer to troubleshoot hardware. Don’t ask a backend dev to write a frontend. Don’t ask a proctologist to look at your cough.

          You simply cannot be proficient at every sub-sub-specialty. That’s why we collaborate and hand the ‘my computer gets hot’ problems to the hardware people. The alternative would be only moderately useful generalist.

          • IrateAnteater@sh.itjust.works
            link
            fedilink
            arrow-up
            16
            ·
            2 days ago

            I’m not asking everyone to be able to become a hardware specialist, but if you can’t even figure out “my computer gets hot” I’m not going to be able to trust anything you do. Identifying a heat issue does not take a rocket surgeon.

      • TwanHE@lemmy.world
        link
        fedilink
        arrow-up
        3
        ·
        2 days ago

        If it was an nvme ssd i could almost believe it. Some come with totally underspecced heatsinks

    • MajorHavoc@programming.dev
      link
      fedilink
      arrow-up
      29
      ·
      2 days ago

      has Elmo only hired people who claim that they can “wrangle data” and say “yes” ?

      There’s two issues going on:

      1. Elmo’s sociopathic approach to laying people off is public knowledge, and top experts have the luxury of not even applying for his jobs.
      2. Elmo’s ability to judge engineering talent has likely been wildly exaggerated thanks to how he has successfully bought organizations full of talented people, in the past.
      • Kane@femboys.biz
        link
        fedilink
        arrow-up
        23
        arrow-down
        2
        ·
        2 days ago

        Hey! Thats offensive to 19-25 year olds, there are many who just finished college/university and are more than aware.

        They’re just role playing like in movies, with no idea of the consequences.

        • entropicdrift@lemmy.sdf.org
          link
          fedilink
          arrow-up
          13
          arrow-down
          2
          ·
          edit-2
          2 days ago

          How on earth is it offensive to say they’re “not experts”? They’re not prodigies with PhDs. These specific young men are just technical enough and ideologically aligned.

          • Kane@femboys.biz
            link
            fedilink
            arrow-up
            5
            ·
            2 days ago

            Except they’re not, as you will know their tweet would be false after your first year of any technical (IT oriented) education.

            • Zorsith@lemmy.blahaj.zone
              link
              fedilink
              English
              arrow-up
              7
              ·
              edit-2
              2 days ago

              First year? That shit is like A+ cert level knowledge or below, and A+ is damn near worthless. They would know that in the first few hours of a study guide

              • Kane@femboys.biz
                link
                fedilink
                arrow-up
                1
                ·
                2 days ago

                I was being generous when you consider the people in school who somehow pass, even when they don’t know a thing 🥲

              • Kane@femboys.biz
                link
                fedilink
                arrow-up
                2
                ·
                2 days ago

                Apologies, if I came over as hostile. I did not get your meaning through text.

              • grue@lemmy.world
                link
                fedilink
                arrow-up
                0
                ·
                2 days ago

                Even then, no. These were all obviously nepotism hires who would not have otherwise qualified.

                • entropicdrift@lemmy.sdf.org
                  link
                  fedilink
                  arrow-up
                  1
                  ·
                  2 days ago

                  Meh, some of them won some hackathons and scholarships, it’s pretty clear they’re otherwise at least somewhat bright but they don’t have any relevant domain knowledge.

                  In other words, the type of person most likely to be prone to hubris and catastrophic failures.

          • Windex007@lemmy.world
            link
            fedilink
            arrow-up
            2
            arrow-down
            1
            ·
            2 days ago

            Your original comment was ambiguous as to if being an “expert” and “being 19-25” are mutually exclusive.

      • Jo Miran@lemmy.ml
        link
        fedilink
        arrow-up
        11
        ·
        2 days ago

        There is nothing wrong with being 19-25. There’s something wrong with being wholly incompetent.

        • ploot@lemmy.blahaj.zone
          link
          fedilink
          English
          arrow-up
          7
          ·
          2 days ago

          There’s not really anything wrong with being incompetent, so long as you have the humility to admit it and learn from people who know better, and try not to cause harm. That’s not Musk’s minions though.

          • Jo Miran@lemmy.ml
            link
            fedilink
            arrow-up
            4
            ·
            2 days ago

            I think it’s important to differentiate incompetence from ignorance. Ignorance is not knowing. Incompetence is not being able to fulfill the requirements for your assigned task. If you cannot fulfill the requirements for your given task, then you should not be given said task.

      • snooggums@lemmy.world
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 days ago

        Bunch of 1337 hax0rs script kiddies who don’t understand anything but they suck elon’s balls or something idk.

    • Jax@sh.itjust.works
      link
      fedilink
      arrow-up
      5
      ·
      2 days ago

      You have to understand that the average Trump voter probably knows everything they know about computers from watching the ‘wacky-zaney hacker with personality issues/quirks’ “hack” into things by tippity tapping their fingies on a keyboard in your average copaganda performance.

      This is something those types of people will believe.

  • jkercher@programming.dev
    link
    fedilink
    English
    arrow-up
    27
    ·
    2 days ago

    60k rows of anything will be pulled into the file cache and do very little work on the drive. Possibly none after the first read.

  • Tiefling IRL@lemmy.blahaj.zone
    link
    fedilink
    arrow-up
    91
    ·
    2 days ago

    60k isn’t that much, I frequently run scripts against multiple hundreds of thousands at work. Wtf is he doing? Did he duplicate the government database onto his 2015 MacBook Air?

    • 4am@lemm.ee
      link
      fedilink
      arrow-up
      56
      ·
      2 days ago

      A TI-86 can query 60k rows without breaking a sweat.

      If his hard drive overheated from that, he is doing something very wrong, very unhygienic, or both.

    • IsoKiero@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      11
      ·
      edit-2
      2 days ago

      Don’t know what Elmos minions are doing, but I’ve written code at least equally unefficient. It was quite a few years ago (the code was in written in perl) and I at least want to think that I’m better now (but I’m not paid to code anymore). The task was to pull in data from a CSV (or something like that, as I mentioned, it’s been a while) and it needed conversion to XML (or something similar).

      The idea behind my code was that you could just configure which fields you want from arbitary source data and on where to place them on the whatever supported destination format. I still think that the basic idea behind that project is pretty neat, just throw in whatever you happen to have and have something completely else out of the other end. And it worked as it should. It was just stupidly hungry for memory. 20k entries would eat up several gigabytes of memory from a workstation (and back then it was premium to have even 16G around) and it was also freaking slow to run (like 0.2 - 0.5 seconds per entry).

      But even then I didn’t need to tweet that my hard drive is overheating. I well understood that my code is just bad and I even improved it a bit here and there, but it was still so very slow and used ridiculous amounts of RAM. The project was pretty neat and when you had few hundred items to process at a time it was even pretty good, there was companies who relied on that code and paid for support. It just totally broke down with even a slightly bigger datasets.

      But, as I already mentioned, my hard drive didn’t overheat on that load.

    • socsa@piefed.social
      link
      fedilink
      English
      arrow-up
      6
      ·
      2 days ago

      I’ve run searches over 60k lines of raw JSON on a 2015 MacBook air without any problems.

    • vga@sopuli.xyz
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      2 days ago

      I mean if we were to sort of steelman this thing, there sure can be database relations and queries that hit only 60k rows but are still hteavy as fuck.