Dear Michael,
you start your essay by noting what could be reformulated as a tension between three classical arguments in the philosophy of science---Putnam's 'no miracles'-argument, Quine's 'indispensability'-argument, and Laudan's 'pessimistic meta-induction'. The 'no miracles'-argument tells us that theories must get something about the world right, as otherwise, their (predictive) success would be inexplicable (miraculous). The indispensability argument tells us that we must believe in the ontological reality of those elements of our theories which are indispensable to their success---including the mathematical ones. Pessimistic meta-induction, finally, points to the history of scientific theories, and concludes that, no matter how successful they are, our current theories are most likely wrong, and will eventually be replaced by better theories.
Together, this forms a conundrum: we have to believe in entities postulated by theories, that nevertheless we have reason to expect will turn out to be wrong!
You argue for a related conclusion, but by a different route. Unfortunately, I'm not entirely sure I understand your argumentation (or if I do, whether it's correct). You say that the Gödelian phenomenon points to facts true only about our theories; but the common view is rather the opposite---namely, that it tells us things about the models of theories that the theories themselves don't. Thus, the fact that Peano arithmetic doesn't prove the Gödel sentence nor its negation tells us that there are models of the Peano axioms---structures such that the its axioms come out true---in which the Gödel sentence holds, and such in which it doesn't. Hence, it is independent from the axioms.
In other words, the Peano axioms don't have a single 'subject matter', but apply to different subject matters, which will not be isomorphic to one another---and the Gödelian proof tells us furthermore that we can't repair this fact: there will always be multiple subject matters for a given theory. For the Peano axioms, one such subject matter is the natural numbers N: as the 'standard model' of the Peano axioms, the Gödel sentence will be true for this structure. But another subject matter is the natural numbers extended by 'transfinite' elements, which can't be written in terms of a finite number of applications of the successor-function, and for which the Gödel sentence may be false (while all the Peano axioms still apply perfectly well).
This is, in principle, all perfectly ordinary. Consider the field axioms---the common properties of addition and multiplication: associativity, commutativity, existence of an identity-element, existence of inverses, and distributivity of multiplication over addition. These axioms leave a number of questions one might have unanswered---such as, for any element e of a field, does there exist an element e' such that e' * e' = e (i. e. is there a square root for every element)? There are fields where that's the case (the real numbers), and fields where it's not (the rational numbers, where e. g. no element exists that yields 2 when multiplied with itself).
Consequently, there are multiple, inequivalent 'subjects' of the field axioms. But that's no matter for concern. We could imagine extending the axioms such that the matter of square roots, for example, is settled, thus further 'narrowing down' the subject matter.
But what Gödel taught us is that there's no end to this process, for sufficiently expressive theories: no matter how much we try to narrow down the subject, we always find that we can substitute alternatives to what we originally had in mind, while still fulfilling the axioms we have postulated.
Or, as another example, take the axioms for a group, and an Abelian group: the latter are just the former, augmented by the axiom that the group operation must be commutative. Hence, both the real numbers with multiplication and 2x2-matrices with real entries under matrix multiplication are groups (fulfill the group axioms), but only the former is an Abelian group---the subject matter has been narrowed down thanks to the addition of the commutativity-axiom.
Robinson- and Presburger-arithmetic are subtheories of Peano arithmetic, with less expressive power. Consequently, everything true in either is also true in Peano arithmetic, and in every model thereof. Additionally, Robinson arithmetic is, in fact, also subject to the incompleteness theorem, while Presburger arithmetic isn't---the latter doesn't include multiplication, and thus, one can't formulate a Gödel numbering scheme. Thus, Presburger arithmetic simply lacks the power to express the Gödel sentence.
So I'm not quite sure what to make of your argumentation that Gödelian phenomena tell us something merely about the theory, and not its subject matter. To me, to the extent that all three theories of arithmetic are about the same subject, they capture it in greater or lesser detail---like the field axioms are as 'about' the real numbers as they are about the rational numbers, thus merely not capturing the detail of whether there are square roots for every element, and like the group axioms don't capture the detail about whether the group operation is commutative.
In other words, while Presburger arithmetic is complete as a theory, it doesn't describe all of the properties of the natural numbers fully. Furthermore, all of the models of the Peano axioms are also models of Presburger arithmetic, including the non-standard ones. Robinson arithmetic has a model in the form of the set of integer-coefficient polynomials with positive leading coefficient. So Peano arithmetic will have fewer models, because it characterizes them more fully; but by Gödel's theorem, no complete characterization is possible.
So I would say that, rather than the Gödel sentence relating to the theory instead of its subject matter, it effectively shows us that there is always more than one possible subject matter, and the theory doesn't suffice to adjudicate between them. But perhaps I'm misunderstanding you?
Anyway, I wish your essay the best of luck in this contest.
Cheers
Jochen