Answer
Inhibitory circuits have many roles in the brain. They help produce oscillations and regulate the neural gain function. See this scholarpedia article for more on these topics.
Inhibition is also critical for direct computations. Computational inhibitory circuits are divided into two kinds: feed-forward and lateral circuits.
Feed-Forward Inhibitory Circuits
A diagram of a feed-forward inhibitory circuit appears above, with excitatory neurons in green, with pyramidal cell bodies, and inhibitory neurons in magenta, with oblong cell bodies.
The bottom layer receives input. This input could be direct sensory input or it could be the output of an upstream circuit.
For sake of a concrete example, let's say these neurons are somatosensory neurons detecting the deflection of the skin.
The responses of these neurons are highly correlated: the skin is elastic, so pressure at one point causes a broad deflection. This kind of a correlation in activity is sub-optimal from an information-theoretic point of view and from an evolutionary standpoint: precious energetic and computational resources are wasted.
Feed-forward inhibitory circuits reduce the redundancy in representations by "subtracting out" shared elements.
Imagine we push the skin down at exactly the center of the receptive field of the center neuron. This will cause a large response by that neuron and a slightly smaller response in the two on either side.
The activation of the central neuron engages its two paired inhibitory neurons, which synapse onto the next layer of neurons. They "veto" the response of the left and right sensory neurons, preventing them from activating their downstream partners.
So a verbose message on the bottom layer: "the skin is deflected by X at point A, 2X at point B, and X at point C", has been simplified into "something is deflecting the skin at point B". This shorter message can be passed up to higher layers, which can reconstruct the full message by modeling the effects of deflection on the skin: a deflection at point B causes neighboring points A and C to be deflected by half as much. The deflections at points A and C have been explained away by the deflection at point B.
Appropriate tuning of the weights can ensure that if, say, pressure is applied at point A and point B, both of those neurons activate.
Lateral Inhibition
At first glance, the circuit above, representing a lateral inhibitory circuit, appears identical to the feed-forward inhibitory circuit. The crucial difference is that in a lateral circuit, inhibition is direct at a neuron's peers, rather than at its output layer.
This means that the activation of one neuron in a group leads to the deactivation of other neurons in that group. For this reason, the circuit implemented by lateral inhibition is sometimes called the "Winner Take All" circuit.
The same purpose of redundnancy reduction is achieved, but with some slight differences.
While feed-forward circuits leave the information in the lower layer unchanged, lateral circuits get rid of this information. The behavior of lateral circuits is good for efficiency, but it is bad if that information is needed for another computation.
Furthermore, lateral circuits solve a slightly different problem. Feedforward circuits directly subtract activations: the inhibitory circuits act like a kind of "model" for how much of one neuron's output is explained by another.
Lateral circuits, on the other hand, are useful when exactly one neuron should activate, as in one method for creating an invariant representation. Homeostatic mechanisms will cause this circuit to settle on some fixed number of active neurons, no matter how much input comes in. This process is a form of constraint satisfaction, a broad class of problems from computer science.
Both "explaining away" and "constraint satisfaction" are critical components of sparse coding, a particularly powerful algorithm for redundancy reduction in neural circuits that uses lateral inhibition.