Scott Mlyn | CNBC

If there’s one thing we can take away from this week’s stock market dive that sent the Dow down more than 866 points and 5 percent in five days, it’s that American’s are feeling a little jittery. After five years of economic uncertainty, it finally feels as if things are back on track, yet many people are wondering whether or not the terrible times are truly in the past.

On the surface, everything seems to be humming along nicely. The S&P 500 is up about 30 percent over the last two years, the U.S. jobless rate is at a six-year low, housing starts have climbed 9 percent year-to-date, and other economic indicators are pointing to good times ahead.

It’s been pretty good so far, said Sheryl King, Roubini Global Economics’ senior director of research. She thinks America’s GDP will see about 3 percent growth in 2015, but only if some of the risks that people are concerned about don’t materialize and derail the recovery.

A number of things could trip growth up. Here are five main issues keeping economists up at night.

1. Global slowdown

On October 9, Germany reported a 5.8 percent decline in exports, and it didn’t take long for its GDP forecasts to be revised.

The Eurozone’s largest economy was expected to grow by 1.8 percent, but this week the German Economy Ministry slashed its forecast to 1.2 percent, blaming crises abroad and moderate global growth.

If Germany falters. the entire region could be sent back into recession, said Andrew Kenningham, a senior international economist with Capital Economics, and that has a lot of economists worried.

Then there are China’s economic struggles. The once-booming economy has slowed over the last two years—it went from 10 percent GDP growth in 2010 to an estimated 7.4 percent this year—and it could get worse, said Mark Zandi, Moody’s Analytics’ chief economist.

While the country is trying to address some of its growth problems through monetary and fiscal stimulus, it would be a problem if they misjudge, he said. If China does get it wrong, commodity prices could decline further, and that would hurt everyone.

Add unrest in Hong Kong, a Ukraine and Russian war, fighting in the Middle East and even Ebola to the mix and it’s clear that the global economy is on shaky ground.

A global economic slowdown would have an impact in two main ways: It could hurt exports, and it might make people nervous enough to pull their money out of the stock market, Zandi said.

Exports account for about 10 percent of America’s GDP, King said, so while a slowdown wouldn’t torpedo growth, it will still hurt it.

Exports could start to decline because the terms of trade will become much less favorable, she said.

2. Rising long-term rates

People are so focused on how a rise in the Fed Funds rate will impact the economy that many aren’t thinking about the bigger danger: a sharp rise in long-term yields.

Unlike the overnight rate, which is controlled by the Federal Reserve, the market dictates long-term rates. If investors start getting antsy about inflation and whether or not Janet Yellen will up the Fed Funds rate, 10-, 20- and 30-year interest rates could jump, Zandi said.

A large rise would make 30-year mortgages more expensive, which could then derail the housing market’s recovery.

If this happens, we won’t get the lift from housing to support continued strong growth, he said.

A quick rate rise could also negatively impact business borrowing, according to Kenningham.

Companies typically borrow for long periods of time, so it’s long-term rates, and not the overnight lending rate, that affects them most.

If interest rates rise too quickly, then borrowing could become more expensive and that would impact investment-related business decisions, he said.

While he’s not convinced this will happen, he would start worrying if the 10-year Treasury yield rose above 4 percent over the next two years.

That said, the 10-year yield fell the most since 2009 this week, and many traders pushed back expectations for a raise in interest rates from September 2015 to December 2015, according to federal fund futures data compiled by Bloomberg.

Zandi, though, still thinks the key threat to strong economic growth over the next two to three years is a jump in long-term rates.

Long-term rates can rise as fast as they have fallen in recent days, he said.

3. Lower oil and lower inflation

Since the end of May, the price for West Texas Intermediate crude has fallen by about 20 percent to below $80 a barrel at press time. While that hasn’t affected the American economy yet, it could cause problems if prices drop further.

In some ways, a lower price per barrel of oil is good for consumers—gas prices will fall, leaving more money in people’s pockets—but it could have a negative affect on inflation, Kenningham said.

Gas prices have always been intertwined with inflation, so it’s no surprise that we’ve seen it fall over the last couple of months. In June the inflation rate was at 2.1 percent. That fell to 2 percent in June and to 1.7 percent in August.

While that’s still close to the Federal Reserve’s 2 percent target, those numbers could get worse if oil prices keep dropping.

It’s not a problem if inflation falls to 1.5 percent, but it would be an issue if it dropped to, say, 0.3 percent, Kenningham said.

Read More Oil extends rout as economic worries mount: WTI below $80

If that happened, the U.S. could find itself inching toward a deflationary environment.

It can lead to lower spending, because people can postpone expenditures thinking that prices will fall further, he said.

4. Rising inflation

Another problem would be inflation rising too quickly. If food, gas and other goods get too expensive, then consumers could become stretched, King said.

Ideally the Federal Reserve wants to see a gradual rise in core inflation—from about 1.5 percent today to 2 percent by 2016— accompanied by a slow rise in the overnight rate. If inflation jumps, though, the Fed may have to raise rates earlier than expected.

According to King, The question is, What happens if inflation starts to accelerate more than expected? Does the Fed then get nervous about it getting out of hand and then make larger raises in the Fed Funds rate?

The worry here is that the Federal Reserve will have to raise rates faster and higher than planned to curb rising inflation. They then may end up essentially chasing inflation and keep increasing and decreasing the rate.

We saw this in 1994, King said. The Fed was raising rates by 60 basis points in a single meeting and then had to reverse course because the economy looked like it was slowing down.

Rising inflation can cause two problems: It can make it hard for companies to set their prices for the future, and it’s also difficult to get it under control without aggressive monetary policy, she said.

Fortunately, the Fed is keeping a close eye on things.

If the economy is strengthening and can withstand higher interest rates, then the Fed will move rates higher, she said. They’re watching; they’re not on autopilot.

5. Falling stock market

While a falling stock market does not mean the country’s economy is in trouble, there’s never been a recession without a plummeting market.

As of Wednesday, U.S. small-cap and mid-cap stock indexes were in corrections (declines greater than 10 percent), while the Nasdaq 100 and S&P 500 Index were nearing correction territory, down roughly 8 percent.

While that may seem significant, Americans shouldn’t be too worried, said Zandi. If we see a 10 percent to 20 percent correction, then the economy could be in trouble, but what we’re seeing now is a garden variety correction, he said.

This decline will prove more or less temporoary, he said.

However,he does make room for black swan events—an ebola pandemic and ISIS conflict, to name two—that could sink the markets further.

A more significant falling stock market could hurt wealth creation, which would then slow down consumer spending. It also makes it more difficult for businesses to raise capital, which can harm investment spending. Finally, if the market falls, the public could start to panic.

It will affect the collective psyche, Zandi said. And no economy in the world is more sensitive to the ups and owns of the stock market than our economy.

Saying all this, things would have to take a significant turn for the worse for these risks to materialize. Zandi thinks that the economy is performing well and that the good growth should continue.

However, he’s still keeping these potential risks top of mind.

People tend to think their own country is in better shape than everyone’s else and that the risks are outside of their own border, Zandi said. We may be guilty of that in the U.S, so be careful not to lose sight of the things that could be problems.