There’s probably no consensus definition of research. So I’ll take the entry from Wikipedia as a fairly safe summary: *“Research can be defined as the search for knowledge, or as any systematic investigation, with an open mind, to establish novel facts, solve new or existing problems, prove new ideas, or develop new theories.” *Search, knowledge, investigation, mind, facts, problems, ideas, theories; these are the concepts that anchor our notion of research.

I have a growing tendency to sidestep these and instead think: invention, construction, instrument, equipment, technology, and tools. It’s not that I think the conventional concepts aren’t important. I just think it’s usually more efficient to look at the instruments and technology used in research before taking the time to consider its ideas, theories, hypotheses, facts, or evidence. And that’s especially true if the research involves any kind of human behavior or social phenomena.

The human world is an extremely messy place. Science is about dividing, analyzing and reassembling. There are far more ways to divide, analyze and reassemble the human world than are possible to evaluate and compare. Clearly some ways are better than others, so how do we decide which? Science is also about evidence: connecting hypotheses forward and backward to observable data. So perhaps we could evaluate science and research based on its data and evidence. Yeah, we could and should. But there’s even more data out there than there are theories and ideas, and comparing huge surveys and other datasets quickly becomes only marginally more valuable than comparing the ideas and theories behind them.

So here’s what I tend to do. Instead of asking what a researcher is hypothesizing or what data they’re presenting, I look at HOW they acquired and developed their data. If they conducted a survey and ran a regression on it, I tend to not look much further. I only have a limited amount of time. How can I trust that this particular survey and regression are any more informative than the hundreds of thousands of other surveys and regression-based studies that have been done and that I haven’t had time to look at? Do I just trust the apparent coincidence that brought me into contact with that particular study as opposed to all the others out there?

The barrier to entry that I use to determine whether research is worth considering further is whether the researcher developed a new tool for observing the behavior she’s researching, or whether she used an older tool in an innovative way. It’s difficult to invent or develop research technology and tools. Researchers in social science disciplines aren’t usually taught that we have to do that kind of thing. It’s common to think we’re supposed to spend our time in the world of ideas. Nonetheless I say our real job is to import goods from that world into the world of hard practical realities. In his last post Schaun pointed out that good research needs hard walls of reality to approach and overcome. I think *how *we approach those walls is just as important as whether they are there. Especially for behavioral research, it’s really, really hard to approach such walls without really innovative tools.

Here are some examples.

Iain Couzin is a computational biologist who studies adaptive collective phenomena in animal groups – the complex and coordinated collective behaviors that result from social interactions among individuals. He presented several great streams of research in a talk he gave this past fall, most of which were characterized by innovative uses of technology. For example he helped develop video-technology to track the head direction (as a proxy for gaze direction) of individuals in large crowds. Then he used that technology to observe the perception and movement patterns that emerge in response to interventions like an individual engaging in surreptitious behavior (in this case, surreptitiously filming something in the crowd). So for instance, how do crowds respond when they start to realize that something isn’t quite right?

There are cell-phone studies that track movement and reveal routinized behavior over time and which reveal social network information. Mobile phones and other kinds of information and communication technologies are being used more and more to observe large-scale social behavior. Innovative researchers have used internet traffic to investigate the spread and contagion of ideas and information. Others have examined the evolution of language using around 5 million books stored by Google. Using a robot capable of simulating facial expressions and filmed interaction, researchers have been able to explore how people use physical cues and facial expressions to evaluate the trustworthiness of others. Check out the MIT Media Lab and the range of research projects they’re involved in, almost all of which are characterized by innovative uses of technology as a way of observing and recording behavior.

What differentiates the examples above from the majority of the research that’s out there is that each brings a relatively new tool to the table, or uses an older tool in a relatively new way. The questions, hypotheses, conclusions, and even the actual data they use are all conditional on the equipment they used to get their data. The people behind the research are as much inventors and engineers as they are researchers.

I arrived at this position somewhat intuitively, mostly as something I built up while trying to deal with the massive amount of information one could try to assess and use. But it’s not really a new position to take. Just today a friend shared this article with me. It was written in 1994. The author argues that “what is fundamentally lacking in the social sciences is a genealogy of research technology, whose manipulation reliably produces new phenomena and a rapidly moving research front.” *In the text he puts in this clever way: *“What guides them is on a nonintellectual level, a sense of what kinds of physical manipulations have resulted in interesting phenomena in the past, and what sorts of modifications might be tried that will produce yet further phenomena…Research scientists lead a double life: as intellectuals in the game of argument for theoretical positions and as possessors of a genealogy of machines.”

And if you think about it, this isn’t just about the last decade of computer- and ICT-driven development. B.F. Skinner is one of the most famous psychologists of the 20th century, and he’s known primarily for the “Skinner box” – a tool for measuring behavior that wasn’t even developed to measure human behavior. There’s also the Implicit Association Test, a tool developed in the late 1990s to make implicit associations and attitudes observable. And it’s not just about the tools for observing behavior. Invention and innovation also occur within the tools used to analyze the resultant data. The R Project for Statistical Computing has been built by an entire research community devoted not just to using the same methods over and over, but instead to constantly developing new methods that in turn make it worth trying to observe behavior in yet newer ways.

I’m not denigrating theory or the importance of the questions and hypotheses posed by research. Nor am I calling for a mindless or uncritical use of technology in research. But here are some questions I think are worth considering:

If the tools we’re using to observe behavior have been around for decades, if not centuries, and if the data we’re using is just a new sample of the same kind of data that’s been around for decades, what are the chances that the difference between what we’re doing and what has been done over and over in the past is the cleverness or originality of our questions or hypotheses or conclusions? What gives us confidence that what we’re doing is worth doing, and that it’s not just our unconscious biases and inherent self-preference that’s making us satisfied with our own research? If the only way to answer the research questions we pose is to ask people to tell us what they think about something (for example), then what makes us think we’ve thought of a question that hasn’t been asked over and over again? Or if it hasn’t been asked, what gives us confidence that it’s worth asking?

It’s a question that I’ve had to ask myself about my own research over and over again. A year or two ago I wouldn’t have known how to build anything, and the only way I knew how to use a tool like a keyboard was to type in English or Turkish. I just don’t think that’s good enough anymore. So now I’m spending my evenings in computer science and physical computing courses. Those are the tools I’ve decided to learn how to use, so that eventually I can build my own. I think of it like this: Good research needs hard walls. When it meets them it should inform how to go through, over, around, or under them. All of that involves tools. We should help design, build, and test those tools; not just sit back and hope they get built one day by someone else. And when we’re looking to be informed by others’ research, it makes sense to spend most of our time on the research that has demonstrated originality and value as much in how it has observed behavior, and made behavior observable, as it has in the cleverness or originality of its theories and hypotheses.

If you want to comment on this post, write it up somewhere public and let us know - we'll join in. If you have a question, please email the author at .