& is used variously in PHP to denote References (see this manual section), but it is misleading to think of it as being an operator in its own right. This is why some people prefer to write $foo =& $bar rather than $foo = &$bar - it means the same thing, but emphasises that the "reference-y-ness" is a property of the assignment, not of the variables.
In some programming languages, such as C or C++, you can "get a reference to" a particular variable; the resulting value can be passed around as a distinct entity, and then "de-referenced" to find out where it points. This is not what PHP references are.
Instead, all PHP variables are actually references to an internal type called a zval. You cannot directly manipulate zvals in PHP, and nor can you make extra layers of indirection - every variable is a reference to a zval, and that's it. (See caveat: objects below.)
What an assignment-by-reference ($foo =& $bar), a pass-by-reference (function foo(&$bar) { ... }), or a return-by-reference (return &$foo) do is tell PHP that you want two variables to point at the same zval. Note that you are not pointing one variable "at" another - they are both equally "real", and calling unset() on either will leave the other completely untouched.
Caveat: objects
It is often misleadingly said that since PHP5 objects are "always passed by reference". The truth is that they have an extra layer of indirection, where the zval is itself a pointer to a particular object. This gives us three different things we can refer to: the variable, the zval it points at, and the object that that points at:
// Create an object, and point three variables at it in different ways:
$foo = new stdClass;
$bar_by_value = $foo;
$bar_by_ref =& $foo;
// Change the object: updates the object shared by all three variables
$foo->value = 42;
// Change the value (zval) of $foo: will update $bar_by_ref,
// but $bar_by_value still points at the original object
$foo = 42;
// Change the variable itself: will never affect $bar_by_value or $bar_by_ref
unset($foo);
