What is a fact, physically?
A fact is a record that survives every comparison anybody is entitled to make, and it is the smallest piece of objective reality there is. Observers meet at shared boundaries called seams, and what two of them can write down at a seam is a comparison: how far apart their readings are, and which way round. Collect every such entry and you have a list anyone can check by walking up to the seam it names. Everything else is somebody's private state. A reading becomes a record when it stops changing, which happens only once the disagreement around it has been repaired, so agreement comes first and memory second. Courts run on the same maxim, that what is not in the record is not in the world. The world is whatever the agreed records jointly name.
Also in Chapter 10: Why Doesn't It Matter Who Goes First?, Chapter 13: What Is One Piece of Reality Actually Made Of?.
This answer comes from OPH, Observer Patch Holography. OPH is a rigorous, consistent and complete description of the unified, self-creating computational Universe we inhabit. Its consistency is machine-checked: the fixed point at its core is constructed explicitly, and a theory with a model contains no contradiction.
To understand OPH and check every step yourself, start with the free book Reverse Engineering Reality. It builds the theory from the ground up, and the OPH Notebook answers your questions while you read.
read the free bookask the notebookor ask sage:read chapter 11: What Does Everybody Actually End Up Holding?watch a universe make its own particles