Style Guide cannot begin with a period and cannot be case sensitive.

Currently, it appears that the Style Guide functionality only works “within” any given sentence; meaning that a rule starting with a period in order to ensure it only runs at the start of a sentence will not function.
Example: “. That is” → "; "

Furthermore, as what I am trying to add would not function properly for the first sentence in a paragraph, I was trying to include a case-sensitive version; but this does not appear to be possible.
Example: “That is” → “” (The description here would suggest avoiding this as a paragraph start).

That’s indeed a known limitation. Unfortunately, I can’t think of a workaround for it.

Couldn’t a more advanced system be made available to advanced users, using something like regex?

I am guessing that the core systems used to write rules for LanguageTool allow for crossing sentence lines and case-sensitivity, so it could theoretically be possible to open up some of that functionality to users, no?

There are indeed more complex ways to write rules (documented here), but we don’t want to make them available to end users in the cloud due to the complexity and the need for support that this would cause. If you’re interested in a local, rule-based checking: there’s documentation for that here.