Rock Island State Park
Rock Island State Park straddles the county boundaries of White and Warren. The less busy, more hiking available, best canyon and waterfall views are all best observed from the north side in White County.
MapsThe canyon you walk through is formed almost entirely in the Fort Payne Formation, so look for freakishly huge crinoids in the boulders that are too large for people to carry off in their pockets. Burrow casts are also visible in many of the rocks. Surely in times prehistoric there were geodes-=a-plenty to be found there, but the hoards of tourists have long since stripped that resource from the area.
WaterfallsAs you approach on a warm day with the windows down you can hear the roar of Twin Falls (Tennessee's largest waterfall by volume) before you can see it. It is effectively a man-made waterfall that results as a portion of the Collins River is pirated through conduits in the cliff as a result of Great Falls Dam raising the water table.
From the northern approach one can go left to go upstream and see a handful of waterfalls including Lower Great, Great, Ice Hole (lol), and Curtain Falls. Before the days when the park closed the gorge to early morning access, this was a secret sunrise spot for me and gave me many good photos. Downstream there is a lingering view of Twin Falls, and there is Little Falls, which is a triple natural arch, travertine formation, and waterfall (useful when you're going to landform bingo).