IQ and Motivation

So one of the most interesting items I came across while looking into the concept of intelligence was this blog post by intelligence scholar Earl Hunt. Says Hunt:

The statistics are pretty clear. Tests of general cognitive ability are by far the best tests for predicting performance in both academia and the workplace. If a job is cognitively demanding, the correlation between test performance and job performance will be somewhere in the .40-.55 range. This is about twice as high as any tests of personality or motivation

…BUT it’s also pretty clear that people do not believe this. Over and over, I read and hear statements like “Personality is much more important than intelligence” or “Motivation is the important thing.” or ”
I knew someone who had real good (bad) test scores and they did as real bad (good) job.”…

When people reason about things, their personal experiences have much more weight than abstract statistics. AND, second point, we live in a cognitively segregated society. (The reason is at least partly because we live in an educationally segregated society.) To see what I mean, ask yourself how many of your friends…people you deal with at least once a week, and in a setting that is not restricted to formal exchanges, like a passenger-bus driver exchange, have markedly different educational backgrounds than you do?. I would bet that not very many do. (This is not quite so much the case if you are in the military, but that’s  a special situation.)

Put these two tendencies together, and you see that most people do not get to observe the problem solving behaviors of people whose intelligence differs very much from their own. This is certainly true of professionals, college professors, executives, etc….The people who do the talking. Because the typical person sees only a small bit of the range of intelligence that is actually out there, the importance of intelligence…in the big picture, in the whole population just isn’t appreciated.

So IQ matters more than we realize, and more than motivation and personality in the grand scheme of things, at least according to Hunt.

And yet here’s a post via Kevin Drum & Tyler Cowen:

On IQ tests, a single standard deviation equals 15 points. So if this research is right, giving people actual incentives to do well on IQ tests (money, for example) has the following effect:

  • Those with low IQs scored 14 points higher.
  • Those with high IQs scored 4 points higher.

In other words, giving people an incentive to do well collapsed the gap between high and low by ten points — and bigger incentives created even bigger effects. These results are based on a meta-analysis of previous studies, not on new research, and metastudies are notoriously tricky to do properly. So take this with the usual grain of salt until these results get replicated elsewhere.

So IQ may be a better predictor than motivation, but the latter informs the former. I wonder what Hunt’s response would be…

Get smart: 3 different methods for finding good stuff online

I’ve been pondering “intelligence” a bit lately and on Friday decided that I wanted to spend some time this weekend reading about it. I was starting from roughly scratch and looking for good resources to learn how academic researchers define, explain and generally think about intelligence. I ended up finding some good stuff but the whole process got me thinking about how we find stuff online. I found resources via three different methods that I want to briefly discuss: social recommendations, search, and feeds. There’s a lot of overlap between these, but bear with me (or don’t – your call).

Social Recommendations

I tweeted and posted on my gchat status about my desire to read some good stuff on the concept of intelligence and research around it and that got a few bites. Mostly it was just friends asking what I meant by it, or how I defined intelligence, to which I had to reply that I was hoping they could point me to the answer to that question. I did get a couple of links pointed out. Some were only slightly relevant, some were a bit better. But in any case, it turned up some interesting leads but not quite exactly what I wanted. I mentioned that this was the case to a friend (one who had provided one of the more interesting suggestions I received) and added that I was pretty used to lackluster response to my inquiries through social platforms. He remarked that he would have guessed the opposite, given the frequency with which I solicit recommendations.

He has a point. I’m not sure why I keep asking the social sphere for recommendations, despite the fact that the results are pretty weak. Maybe I’m just hopelessly optimistic about social media? In any case, with no disrespect to my friends, (who are smart, interesting, helpful, etc.) for whatever reason (shortage of time, lack of interest in my lame preoccupations) inquiring through social networks is seldom helpful for me. My guess is that this is the case for most people, but that it is neglected because “influencers” at the top of the social media food chain have the exact opposite experience. Clay Shirky can get any question he wants answered quickly with a tweet, and he’s the one writing about how effective this stuff is or isn’t. There may be certain spheres within which social inquiry is effective… maybe if my inquiries were more social, more relevant to my audience, I’d receive better response (I’m imagining things like What fun stuff is going on tonight? / What’s the best ultimate frisbee team in Boston / etc. Things that are highly relevant to a large group of my network.

Search, and not just Google

Obviously I did several Google searches immediately to try and find resources. This was kind of limited. “Intelligence” tends to turn up a lot about military intelligence, which speaks to the lack of intelligence in search, or perhaps just to my lack of it. In any case, I found a few relevant items, but not much.

I cracked through a bit by targeting a few “likely suspects”: The New Yorker, The Atlantic, etc. I hit their Archives to further target my search and that turned up slightly better results. One of the best things I came across was via The Atlantic’s archives (a piece from 1990.)

Feeds totally rock

Far and away the best resource I found – a link to a textbook surveying the literature on intelligence, which I then dug up on Google Books – came serendipitously from one of my favorite blogs: Marginal Revolution. Sure, it was just chance that Tyler Cowen happened to link to that book the same weekend I was looking into the subject, but it wasn’t entirely random either. I subscribe to Marginal Revolution because the content is consistently interesting, and so in a way I have tailored my feeds to maximize the likeliness that the content matches my interest consistently.

Moreover – and here’s a plug for RSS readers over Twitter – my feeds amount to a customized trove of content matched to my interests, and ripe for search. If I want something reliable on a topic, I can search Google Reader and essentially whittle my search down from the whole web to sources I like and trust. And that’s pretty powerful.

The takeaway for me is just that there are real limits to social search, and tremendous potential to carefully curated content feeds, at least relative to the general weights that I think popular commentators tend to assign to each.

If you’re a huge nerd too and want to learn about intelligence, I’ve tagged all my discoveries here on Delicious. And here’s the link to that textbook. Not light reading but I’ve skimmed a couple chapters and it’s just what I was looking for.

Google won’t feed me my vegetables

I had a post months back called “Who will feed me my vegetables?” about the dangers of social news feeds. Here was the gist:

Consider politics.  Facebook knows I self-designate as “liberal”.  They know I’m a “fan” of Barack Obama and the Times’ Nick Kristof.  They can see I’m more likely to “like” stories from liberal outlets.So what kind of political news stories will they send my way?  If the algorithm’s aim is merely to feed me stories I will like then it’s not hard to imagine the feed becoming an echo chamber.

Imagine if Facebook were designing an algorithm to deliver food instead of news.  It wouldn’t be hard to determine the kind of food I enjoy, but if the goal is just to feed me what I like I’d be in trouble.  I’d eat nothing but pizza, burgers and fries.

This is not just idle speculation. Here’s an entry today at the Google News Blog:

Last summer we redesigned Google News with new personalization features that let you tell us which subjects and sources you’d like to see more or less often. Starting today — if you’re logged in — you may also find stories based on articles you’ve clicked on before.

For signed-in users in the Personalized U.S. Edition, “News for You” will now include stories based on your news-related web history. For example, if you click on a lot of articles about baseball, we’ll make sure that you get a chance to see breaking baseball stories. We found in testing that more users clicked on more stories when we added this automatic personalization, sending more traffic to publishers.

Emphasis mine. In many ways this is obviously useful. But it carries real risks. And I bolded that last line to emphasize the driving force behind these efforts: profit. What you should be reading is nowhere in the equation. Even what you want to read is useful only to the extent that it serves up traffic and ad revenue. Somewhat related…I’m increasingly curious about the possibility for “responsible algorithms” to add a new layer to the web experience for users on an opt-in basis. That’s something I’ll expand on in a future post.

Ideology, pragmatism and sacred values

Ezra Klein had a post this week titled “Obama’s budget is policy, not ideology” that contained this bit:

Obama’s budget is not philosophy…it is the product of a negotiation process, as opposed to an opening bid. It is, in other words, policy. You could argue that this is a philosophy, and that philosophy is pragmatism, but I think that’s getting too cute. This is the sort of policy that night pass and might work.

Ryan’s budget is purer, but it is also more fantastical. It posits the government it wishes were possible, and the policies it wishes would work. It is an opening bid so ideological that it leaves little room for a process of negotiation.

Krugman took issue with this general framing (not just by Ezra) in a post titled “Everyone Has An Ideology.” His basic point:

But I’d also like to register a philosophical protest. There’s an old joke to the effect that you’re an ideologue; I’m just being sensible. The point is that everyone has an ideology — which is another way of saying that everyone has (a) values and (b) some view about how the world works. And there’s nothing wrong with that.

As a philosophical matter I believe Krugman is correct. There is no policy without ideology/philosophy. Yet Ezra offers a relevant distinction between ideology and policy: the former ignores the prospects of an idea becoming law while the latter takes that into account.

Ezra’s distinction is potentially useful, and shows that we tend to define “ideology” and “ideological” in ways other than its purely philosophical designation. I want to offer my own definition of “ideological” in the context of modern politics and then tie it back to a previous post on how we identify values in political arguments.

How I think about ideology in modern politics

To me, a person is “ideological” to the extent that they place value on government’s role in a certain sphere of policy, on one side or another. For instance, if you place value on government not involving itself in the provision of healthcare, that makes you ideological. It may still be that once the facts are considered, you end up supporting a role for government, but your starting point places initial value on finding a solution that doesn’t require government. Your scales for considering the issue come pre-weighted against government involvement.

Or take the issue of taxation. If you place strong value on private property, you might approach taxation with the scales pre-weighted against raising rates; arguments in favor of more taxation thus have a higher bar to overcome. These values are perfectly legitimate; but I call them “ideological” because they approach one of the major questions of modern American politics, the appropriate role and scope of government, as at least partially a question of values. (This can occur on both left and right, though I tend to think the right is more likely to apply values to this particular question.)

It is in that sense that I believe Obama is truly a pragmatist. On the question of the scope of government he is likely to ask “what does the government do well, and what does it do poorly, relative to our other goals”, whereas a more ideological figure would ask “does government do x well enough/poorly enough to overcome our presumptive value for/against its involvement in this sphere”.

Identifying ideology

So what’s the point of all this? I wrote a post a while back on Exposing sacred arguments that touched on something similar. As I wrote then:

I see the identification of sacredness as a crucial challenge in the public sphere, and therefore a crucial challenge within media.

I want to use the definition of ideology that I’ve laid out to give a semi-concrete example of what I mean. Let’s assume that when it comes to healthcare, Obama is a pragmatist in the sense I’ve discussed. He’s worried about welfare, equality, etc. but approaches the question of the role of government in healthcare pragmatically. But does Paul Ryan approach that question in the same manner? Or is it possible that when considering the appropriate role of government in healthcare, Ryan’s scale comes pre-weighted, perhaps even just slightly, against government involvement. Perhaps limited government is a “sacred value”, in a Haidtian sense, to him and to his supporters.

If so, that’s something we, the public, should know. So how do we find out? One way is simply to get the facts straight. If we weigh the arguments on the unweighted scale, find the argument for government involvement compelling in that context, and then see Ryan still favoring less government, that might suggest the existence of a sacred value. So classic fact-checking and analysis can help us back into the identification of values.

But I wonder if we might also do some work on the opposite end, by identifying the values first. And I wonder whether that might help us back into our analysis of the issue. If we can identify up front that Ryan places inherent value on limited government (what I’m calling an ideological approach) then we can treat his arguments about the pragmatic merits a bit differently. Perhaps it means we are less likely to trust certain arguments of his. Perhaps we just bear in mind that in his view arguments against government involvement have a lower bar to clear. In any case, it’s something we want to know and something we may be able to use media to help uncover.

Netflix vs. the market

I wrote a lengthy post last year on how the networked information economy offered potential challenges to the market as the primary form of economic organization. In that post I noted the market’s efficiency at aggregating information through individual preferences and used Netflix of an example of how networks are making information aggregation easier in a way that could potentially challenge markets. I want to expand on that. Here’s what I wrote last year:

Just as the market’s claim to dominance in motivating us is starting to be challenged, some are revisiting its dominance in aggregating information.  Sunstein explores the subject in Infotopia and highlights increasing efforts to aggregate human preferences online, including Amazon and Netflix.  If it’s obvious that we are doing better and better at aggregating information thanks to the Net, it’s less obvious how this might challenge the role of the market.

Imagine that Netflix has a small, set number of a rare movie to rent, and that it’s in high demand.  Who should get it first?  Auction the privilege off to the highest bidder, responds the free market advocate.  And, particularly in a scenario where customers have equal wealth at their disposal, this method has a lot to recommend it.  The market is incredibly efficient at allocating resources under ideal settings.  Tremendous gains in human welfare have been predicated on this fact.  But Netflix is developing sophisticated algorithms to use your preferences for movies you’ve seen to predict what movies you’ll like.  Is it so hard to believe that some day in the future an algorithm could – given the aim of maximizing viewer enjoyment – “beat the market” in determining how to distribute the movie?

I want to articulate this challenge in slightly greater detail. Let’s start small and simple…

There are 100 video customers, each of whom has a token for one free movie. Each customer browses the library of videos and picks out the one they want. They watch it and fill out a survey rating how much they enjoyed it. That’s scenario 1.

Scenario 2: Still 100 customers. This time Netflix’s algorithm, looking at their past ratings, gives them the movie to watch. After watching, they rate their satisfaction.

Which group will be more satisfied? What does it mean if Netflix gets to a point where its algorithm wins out? And how about if we change the example by adding scarcity, as was implied in the quote above? Now there’s only one of each movie and we’re comparing Netflix doling out movies via its algorithm to customers bidding on movies in an auction of some kind.

I’d like to ask for some help in thinking about this.

To my tech friends: obviously a ton of people are writing smart stuff about recommendation algorithms. What should I be reading on this?

To my economist and wonk friends: what do you think of the comparison of these two distribution mechanisms? What am I missing? I’d love help thinking about how you’d actually design the specifics of a challenge. While the comparison makes broad sense to me at the macro level I’d appreciate hearing from someone familiar with the economics of auctions, etc. It’s been a while since I’ve thought too much about basic micro.

Government-funded R&D and innovation

If you follow me on Twitter, Facebook or Gchat you already saw this, but just in case: check out my latest post at The Atlantic Tech on the argument for energy R&D. I’ve had some questions on the specifics of energy and there’s room for a follow-up there, but I wanted to make a very basic point about government-funded R&D as an input in the innovation pipeline. Here’s a taste:

Private actors are excellent at taking a newly proven technology and commercializing it, but they have little incentive to invest heavily in the basic science that leads to those breakthroughs. The benefits of a new innovation inevitably spill over into society; firms are unable to capture 100% of the value of new information. The result is that the private sector will always under-invest in R&D, making government funding essential. Corporate R&D, while important, is nowhere near sufficient…

…The majority of academic R&D is, of course, government-funded. The Internet, GPS, CT scans and countless other technologies on which we currently rely are the direct result of government investment.

Last week, the International Energy Agency’s Clean Energy Progress Report stressed the importance of substantially increasing public clean energy R&D, particularly as stimulus funding ends. Yet here we are, debating possible cuts. Innovation is a mantle claimed by both parties, but no matter one’s philosophy of government, any serious account of how it happens must recognize that government funding of R&D is a crucial input. If America is serious about developing the next generation of clean energy technology, it must take ideology out of the equation.

Technology journalism and moral force

One of the things I love about ProPublica, the relatively young investigative journalism nonprofit, is that it states as its focus “stories with ‘moral force.'” But what kind of journalism fits this category? Upton Sinclair? Nick Kristof? My fear is that if we drafted a list of what and who comes to mind, it would contain far too little technology journalism.

I was in a meeting last week with a Massachusetts state senator who was explaining how a major portion of his constituents lack broadband access. They’re dealing with – not kidding – phone service failing when it rains. This is in Massachusetts. I think when most of us think about pockets of the country without broadband we think West Virginia, Mississippi, etc. But even here in Massachusetts the digital divide is all too real.

We have a great wealth of journalism on the consumer and business aspects of technology. If you want reviews of the latest gadget or the inside scoop on Facebook’s latest round there’s no shortage of information. But what about technology from a justice perspective? Certainly these topics get touched on (and there is lots of great academic work on the subject). But my sense is that they are under-served.

It is, of course, quite possible that far more writers and outlets on this subject are out there, and that I just haven’t found them. Is there someone out there today who can claim the mantle of Tech Upton Sinclair?

Yochai Benkler had a line at some point (failing to find the video at present) where, in speaking about his book’s thesis that commons-based peer production had positive moral implications, he said roughly The problem is it takes 500 pages to explain why. Tech journalists should take that as a direct challenge. Technology journalism with moral force is not only possible; it’s socially necessary.