The bookseller's sales in the fourth-quarter were hurt by returns of the device from retailers as well as price cuts put in place in February on some versions of the device to compete with Amazon.com's Kindle franchise.
The company, which has bet its future on staking a claim in the e-books industry, suffered a setback as Nook revenue fell 10.5 percent.
"The weakness should remind investors of the constant need for investments and the intense competition with Amazon and Apple," said Credit Suisse analyst Gary Balter in a note.
But keeping Nook competitive is expensive: the Nook business lost $77 million on revenue of $163.6 million in the quarter.
After getting a bump thanks to defunct rival Borders Group's bankruptcy in recent quarters, the company expects sales at existing bookstores to drop by a low to mid-single digit percent in the current fiscal year.
Barnes & Noble shares closed down 4 percent at $14.63. They are down 43.7 percent from a 52-week high in April.
The company has poured hundreds of millions of dollars into the development of the Nook, including a recent, well reviewed glow-in-the-dark model, whose positive reviews allowed it to garner some 27 percent of the U.S. e-books market in the 2-1/2 years since the reader was launched.
But that compares with 60 percent for Amazon, and in January, the retailer acknowledged slower-than-expected sales of devices such as the black-and-white Nook Simple Touch.
As the black and white e-reader market matures, the battleground is shifting more to color e-readers and tablets that offer more media, and not just text.
"That's where Barnes & Noble has an uphill battle," Morningstar analyst Peter Wahlstrom said. "Amazon has a distinct lead in that area."
Barnes & Noble is counting on a deal with Microsoft Corp and reduced development expenses for Nook to ease losses for the digital business this fiscal year.
The deal, in which Microsoft will invest $300 million and get a 17.6 percent stake in a new subsidiary consisting of the Nook digital business and its College bookstore chain, is expected to close later this year.
Microsoft announced plans on Monday to launch its own line of tablet computers to compete with Apple Inc's iPad.
Barnes & Noble also plans to have Nook devices and digital book stores in some international markets for the holiday season.
Revenue for the quarter was $1.38 billion, up slightly from a year earlier, but below the average analyst estimate of $1.48 billion, according to Thomson Reuters.
The company posted a net loss of $57.7 million, or $1.08 per share, for the quarter, compared with $59.4 million, or $1.04 per share, a year ago.
Excluding one-time items, the loss was 98 cents a share, compared with the average analyst estimate of a loss of 93 cents, according to Thomson Reuters.
Same-store sales at its 700 super stores rose 4.5 percent compared with the year-earlier quarter. But some of that increase was due to more of a boost than the company expected from Borders going out of business.
Same-store sales at its College bookstore chain, which generates nearly one quarter of revenue, fell 2.2 percent during the quarter. But Credit Suisse's Balter said Barnes & Noble has been able to win new contracts.
Copyright Thomson Reuters 2012