PostGhost kept copies of tweets sent by verified users with more than 10,000 followers, the Verge reported on Monday.
Twitter sent a notice to PostGhost that recording deleted tweets was a violation of the service's terms.
PostGhost agreed to shut down, with a detailed note to Twitter that such users are "public figures" that should have their tweets recorded.
"We believe that for such prominent verified Twitter users, the public has a right to see their public Twitter history, whether or not they grow to regret the statements they've made," read a statement from PostGhost.
"We created the website postghost.com to provide the public with a more accurate history of public statements made by the most influential public figures on Twitter."
"We believe PostGhost provides a fairer and more transparent way of allowing individuals to hold public figures accountable than Politwoops, a website that Twitter has recently reauthorised to publish certain deleted tweets," it added.
Recently, Politwoops, a project archiving the tweets of politicians, was shut down. It was, however, revived later after Twitter and Politwoops reached an understanding.
According to Twitter's verification FAQ, Twitter verifies "key individuals and brands" who comprise "highly sought users in music, acting, fashion, government, politics, religion, journalism, media, sports, business and other key interest areas".
Twitter does not "accept requests for verification from the general public".
"In other words, verified Twitter users are not members of the general public - they are influential public figures with the ability to start trends and change opinions," PostGhost said in a statement.
"We're happy to continue a dialog and hopefully work towards a resolution that maintains user privacy while holding public figures accountable," PostGhost told Twitter.
For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.