Just because Denny Carter, and I MyFantasyLeague.com's average draft position data. And, as the table shows above, he's never finished higher than the seventh-best quarterback in a given year.
This year, according to the same data, Ryan's the fifth quarterback off the board. (Fifth!)
So what does this mean? It means that, sure, he's not going to bust for you if he's healthy, but that doesn't really matter at the quarterback position in fantasy football. You actually should be drafting for variance in single-quarterback leagues -- the only players who really are giving you an edge at the position are the elite guys, and we have a fairly large sample size to show us that Matt Ryan, in terms of raw numbers (which are what matter in fantasy football) just isn't that.
Rather than investing a middle-round pick in Ryan, go with later-round upside in players like Ryan Tannehill or Sam Bradford. Though Ryan has better odds in finishing higher than them in fantasy scoring this year, the difference is irrelevant -- the only way Ryan will really be a difference-maker (and you can see the data for this in the streaming quarterback case study that's linked above) is if he makes a jump that he's never made throughout his career.