• OpenStars@piefed.social
    link
    fedilink
    English
    arrow-up
    34
    arrow-down
    1
    ·
    1 month ago

    Except iirc, they aren’t scraping “properly” (read: efficiently at least, setting aside morality for the sake of discussing this component in isolation), and are causing traffic troubles. If only they took the time to install an actual instance themselves then nobody would care in the slightest (again, ignoring the morality part, for now).

    TLDR: they are being dicks about it, bc offering everything we have for free is not enough for them.

    • MrKaplan@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 month ago

      of all the scrapers we see, the requests identified as originating from Meta seem to be well behaved overall. they appear to (mostly) be respecting robots.txt where present and their request volume to Lemmy.World is only averaging slightly above 5 requests per minute over the last 2 weeks. they also don’t spoof their user agents to pretend to be web browsers, or at least I have not seen credible accusations of this happening.

    • Chloé 🥕@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 month ago

      i mean, that’s exactly what they did with threads, and many instances defederated from it because they didn’t want to have their data scraped by meta