fne8w2ah@lemmy.world to Technology@lemmy.worldEnglish · 1 year agoThe Internet is not forever after all: CNET deletes old articles to game Googlearstechnica.comexternal-linkmessage-square16fedilinkarrow-up180arrow-down12cross-posted to: [email protected]
arrow-up178arrow-down1external-linkThe Internet is not forever after all: CNET deletes old articles to game Googlearstechnica.comfne8w2ah@lemmy.world to Technology@lemmy.worldEnglish · 1 year agomessage-square16fedilinkcross-posted to: [email protected]
minus-squareskankhunt42@lemmy.calinkfedilinkEnglisharrow-up11·1 year ago Unfortunately, we are penalized by the modern Internet for leaving all previously published content live on our site Even if this is true, which I doubt, why not edit your robots.txt to disallow them to index it and leave the content up?
minus-squarePuzzle_Sluts_4Ever@lemmy.worldlinkfedilinkEnglisharrow-up2arrow-down1·edit-21 year agoRemoved by mod
Even if this is true, which I doubt, why not edit your robots.txt to disallow them to index it and leave the content up?
Removed by mod