Nikesh Arora, a former Google executive, joined Softbank in 2014 and was named as the next chief executive by Son last year. He invested heavily in SoftBank stock last year.
SoftBank's shares rose 2.6 percent in morning trade.
"Shareholders are welcoming this, I think," said Shigeru Kanno, one of the company's shareholders gathering for the group's annual meeting on Wednesday.
"The share rise today reflects investors' hopes that Mr. Son will be in charge for much longer," said Kanno, a pensioner who said he had been a shareholder since SoftBank went public.
But analysts said Arora's abrupt exit late on Tuesday raised questions over the conglomerate's disparate investment strategy. Arora had been reshaping the portfolio and bolstering the balance sheet in recent months through a series of asset sales.
SoftBank shares have fallen over 15 percent in the past year, mainly on concerns about U.S. wireless carrier Sprint Corp, in which SoftBank acquired a majority stake in 2013.
Arora, who was Google's highest paid executive in 2012, has also been highly paid at SoftBank, raising questions among some investors. He has received over $200 million in compensation over the last two years, though he has sold the $500 million of SoftBank shares he bought last year to Son at a small loss.
"Considering his high compensation he hadn't been able to produce results, so his departure would be accepted by investors," said Yoshihiko Tabei, chief analyst at Naito Securities, citing a slow turnaround at Sprint.
The fourth-ranked U.S. mobile network provider has struggled for profitability in recent years, and for January-March reported a wider quarterly net loss and added fewer subscribers than expected.
To address investor concerns over Sprint, SoftBank has announced three major asset reshuffles in the past month, one of which was the sale of $10 billion worth of shares in the Chinese e-commerce giant Alibaba Group Holding.
On Tuesday, SoftBank said it was also selling its majority stake in 'Clash of Clans' maker Supercell to China's Tencent Holdings.
Arora was credited for those moves, which brought much-needed stability to SoftBank's finances, as well as bold moves into India's fast-growing tech space - prompting at least some on Wednesday to voice concern.
"Mr. Arora did achieve, in terms of finding investment targets that bring returns faster," said Ikuo Mitsui, a fund manager at Aizawa Securities. "They've stemmed the bleeding at Sprint but they've yet to achieve growth. So that's worrying."
© Thomson Reuters 2016
For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.