[hl/std] Prevent StringBuf expand overflow

[yuxiaomao]

The maximum StringBuf size was limited to INT_MAX/3 by the (size* 3) >> 1, this PR allows it to grow up to nearly INT_MAX and throw appropriate exception on too large buffer size.

function main() {
	var bytes = haxe.io.Bytes.alloc(454624961); // ~450 MB
	var hex = bytes.toHex(); // ~900M chars (~1.8 GB), triggers StringBuf expand overflow that result in segfault later
	trace(hex.length);
}

[yuxiaomao]

FYI @ncannasse (fix for toHex)