The nasty cash squeeze that roiled China's banking system in recent weeks could be just a preview of greater instability to come if China's leaders push ahead with liberalizing interest rates and capital controls.

A spike in the interest rates that banks charge for lending to one another sent stocks plunging last month and raised the specter of bank runs as reports circulated online about ATMs with no cash and electronic payments that failed to clear.

But further progress towards de-regulation of China's interest and exchange rates will only increase the challenge for Chinese banks in managing risk, analysts say.

(Read More: China Cash Rates Ease Further as Central Bank Stays Out of Market)

"The recent instability reflects some of the risks with interest rate liberalization that proceeds too rapidly, or more precisely relying excessively on interest rates to achieve objectives," Yukon Huang, former World Bank country director for China and senior associate at the Carnegie Endowment for International Peace, told Reuters.

China's State Council, or cabinet, has said it wants to push interest rate liberalization, and the central bank made a move towards that goal in June last year when it granted commercial banks limited flexibility to vary deposit rates.

One reason that freeing up interest rates could lead to instability is that such a move would unleash fierce competition between banks for customer deposits.

(Read More: China Cash Crunch Already being Felt on the Ground)

"One has to be very careful and think about 'What do we do with banks if they become overly aggressive? What do we do with banks that compete themselves into losses?" Markus Rodlauer, deputy director of the Asia and Pacific Department at the International Monetary Fund, said at a panel discussion in Shanghai on Saturday.

Last month's crunch, which saw China's interbank lending rates soar to record levels on June 20, was engineered by the central bank, which declined to inject liquidity into the market in a blunt signal to lenders to rein in risky credit growth.

The resulting turmoil, which rippled out across financial markets, sending emerging market equities and currencies reeling, was reminiscent of the onset of the global financial crisis in 2008, when money markets froze as banks stopped lending to each other amid fears that many were overextended.

Liquidity Risk

The rise of so-called wealth management products (WMPs), which banks in China market to customers as a higher-yielding alternative to traditional bank deposits, has already increased banks' liquidity risk.

WMP funds are less "sticky" than old-fashioned deposits because investors frequently shift between banks in search of the highest returns.

Banks also face liquidity risks due to the mismatch between the short maturity of WMPs and the longer-term underlying assets. Cash rates spiked last month in part because banks were borrowing in the wholesale market to finance payouts on maturing WMPs.

(Read More: China Banking Regulator Insists Liquidity Sufficient)

Yet the struggles already facing some banks in managing these risks is only a taste of the much more brutal competition that lies ahead.

Outstanding WMPs reached 8.2 trillion yuan ($1.3 trillion) at end-March, China's chief banking regulator said on Saturday. While WMPs barely existed five years ago, the current total still represents only 8 percent of all Chinese bank deposits.

Lifting the cap on deposit interest rates could open up China's entire 99 trillion yuan stock of bank deposits to similar competition.

"Liberalizing interest rates now will inevitably cause deposit interest rates to soar," Huang Jinlao, vice president of Hua Xia Bank, a mid-sized lender, wrote in a commentary on the website of Caixin, a respected financial magazine, in April.

(Read More: Will the Reprieve for China's Lenders Last?)

"Banking institutions may take on excess risk under the pressure of narrowing interest spreads. This could trigger a financial crisis."

Some market watchers take a more sanguine view. Jimmy Leung, lead partner for banking and capital markets at PwC China, says that banks are not only competing with each other, but also with China's vast informal lending sector.

"If we liberalize interest rates more aggressively, the banking sector would actually attract more money from the shadow banking system, or from individuals, to go into the formal banking system, because interest rates would rise, attracting more deposits," he said.

Capital Flight

Interest-rate liberalization could threaten individual Chinese banks, but as long as strict capital controls remain in place, most funds will remain locked inside China, making a system-wide crisis unlikely.

If, however, China proceeds with its goal of making the yuan fully convertible on the capital account, the potential for instability could further increase.

It's easy to imagine how the cash squeeze that roiled markets last week could have snowballed into a full-blown crisis if Chinese depositors -- perhaps responding to online rumors of banks running low on cash -- were not subject to a $50,000-per-year limit on converting yuan into foreign currency.

(Read More: Even the Resilient Yuan Is Feeling China's Pain)

Even with capital controls in place, China suffered mild capital flight in 2012, when fears of an economic hard landing spiked amid China's slowest full-year GDP growth since 1999.

In public, Chinese policymakers downplay the risk of capital flight. China is likely to roll out a new program this year to allow individual investors to make portfolio investments abroad.

"Over the long term, what China mainly faces is foreign capital inflow pressure, not outflow pressure," Sheng Songcheng, head of the research and statistics department at the People's Bank of China, said on Saturday.

But a look at policy details suggests that leaders recognize the risk of sudden outflows.

The individual-investor pilot will likely be subject to a strict quota that renders its impact on China's overall capital flows all but negligible.

And even as Sheng downplayed the risk of capital flight, he also said that China would maintain those specific controls most vital to preventing large-scale outflows, even as less risky items are gradually loosened.

(Read More: China Is Right to tame Credit Growth: Moody's)

"In the short-term we'll promote firms to expand abroad. In the medium term we can relax restrictions on commercial credit that's based on real trade (in goods and services). Only in the long term will we cautiously open real estate, stocks and bonds," Sheng said.

"For short-term debt items where the speculative element is very strong, we may not open them for a pretty long time."

Beyond higher interest rates and capital flight, the cash squeeze also highlights the risk from institutional weakness in China's financial sector, and some analysts say it is this that regulators should focus on how to reduce the likelihood of another shock.

"Dealing with banks in breach of regulations should be done by improving prudent regulations, rather than engineering an interbank credit crunch which could potentially backfire," Lu Ting, China economist at Bank of America-Merrill Lynch in Hong Kong, wrote in a note to clients last week.