No, you do not understand.
Physics needs to start tackling the big questions: what exactly is space, and what exactly is time?
Right now, it's precisely the words "space" and "time" that are meaningless terms when used on the (quite literally) most universal scale. Typically they're the stage on which physics plays out, but stand oddly apart from physics itself. That is not acceptable in this day and age. Very smart physicists are increasingly asking the question: Does time really exist, or is it just a bookkeeping device to organize our thoughts and perceptions? And what is space? To even begin to answer these questions, we must be clear what exactly we mean by time and space. The everyday meanings used for scheduling a meeting doesn't cut it. Space and time figure in countless physical models, but themselves have no explanation. Relativity models the shape of spacetime on a large scale, but says nothing about what is being shaped.
I purpose to address the true mystic here directly. That would be you, old bean. You are part of the problem with physics being stuck in a rut, because you're grafting Newtonian-era constructs onto quantum chimera as if trying to dress your shadow in your Sunday best. When we speak of "space expanding," the question should naturally arise: what precisely is expanding when "space" is expanding?
Well, we must come to grips with quantum entanglement. We see in experiments that particles thousands of miles apart seem to interact (or communicate) instantly. They are "entangled," and at first glance it suggests that information can travel faster than the speed of light. But there is a loophole that saves Einstein: useful information cannot be conveyed by entangled systems by any means we can currently devise or even divine. Nonetheless we see instant effects on a faraway particle when one near at hand is changed in some way, if the particles have become entangled.
It has been posited that what we perceive as "near" or "far" is not really a function of what we understand to be spatial separation (i.e. "distance") in the usual sense*, but rather the geometry of space as our senses perceive it is really like a graph. Here I speak of the graphs of graph theory, not the graphs of a high school algebra class. This conception of the "fabric of space" presents it with edges and nodes, with nodes being particles, and edges connecting particles in accord with entanglement properties. Greater entanglement makes things appear "closer" to our senses, lesser entanglement makes things appear "farther," and things we cannot perceive at all (owing to distance or being hidden by other particles) are things that are not sufficiently entangled with our sense organs for us to be able to discern them. Time fits easily into this scheme, and indeed the scheme meshes with common sense insofar as it's sensible that things we are not in physical connection with are invisible to us and may as well be "infinitely far."
In any case the purpose of a physical model is to find a mathematically useful way to describe a physical phenomenon that elicits more useful information than what raw data about the phenomenon itself can convey. To view spacetime as a graph consisting of nodes and edges representing particles and their entanglements stands a chance of helping connect the large-scale universe with the quantum realm, perhaps to at last attain the holy grail of unifying quantum theory with gravitation. This is the primary motivation for modeling spacetime using graph theory constructs.
Physics fared better back in the age when physicists frequently had a strong background in philosophy. We are in bad need of this now, because modern physics has come to the juncture of matter, energy, and thought. How our very thought may color our perception of reality, indeed how our unexamined habits of thought (e.g. spacetime is a stage for physics that's apart from physics), may very well be what's blocking humanity from achieving the next great breakthrough.
* Here's some of that mysticism: "empty space" creating a separation between objects somehow. Huh! Not good enough for honest scientific sensibilities in the quantum age. What's that "emptiness's" secret for doing so?