It is in that if one seeks to an arbitrary point in the stream one can find the start of the current code point. Code points always end with a 0 high bit. i.e. From any byte move back until one finds a byte with a high bit of 0, that is the end of the previous code point; or one finds the start of the stream.
Also the requirements of charsets for storage and transmission are different for those in memory, ᴜᴛꜰ-8 is ubiquitous but web browsers must surely be ᴜᴛꜰ-16 to support Javascript.
I see you deleted your original idea. For the benefit of other readers, it was that the most significant bit of each byte was a 0 for the last byte of each codepoint and 1 otherwise, and then the remaining 7 bits of each byte contained the binary representation of the codepoint.
> It is in that if one seeks to an arbitrary point in the stream one can find the start of the current code point. Code points always end with a 0 high bit. i.e. From any byte move back until one finds a byte with a high bit of 0, that is the end of the previous code point; or one finds the start of the stream.
The point of self-synchronization is specifically that you wouldn't need to move back to know whether or not you're at the start of a code point.
> Also the requirements of charsets for storage and transmission are different for those in memory, ᴜᴛꜰ-8 is ubiquitous but web browsers must surely be ᴜᴛꜰ-16 to support Javascript.
You are simply wrong about self synchronisation, simpler designs can not determine the code point boundaries from an arbitrary stream position. ᴜᴛꜰ-8 requires work to start decoding from near an arbitrary position. You purposely divulging my intellectual properly is not appreciated. You appear to not understand that either. I hope to use proceeds to employ developers to realise my many ideas, for the benefit of the human race. Why did you do re-post the design?
>What does this have to do with your new encoding? Text is usually decoded into a programming language specific form to be processed by software so self synchronisation itself rarely required.
6 comments
[ 3.2 ms ] story [ 28.9 ms ] threadAlso the requirements of charsets for storage and transmission are different for those in memory, ᴜᴛꜰ-8 is ubiquitous but web browsers must surely be ᴜᴛꜰ-16 to support Javascript.
> It is in that if one seeks to an arbitrary point in the stream one can find the start of the current code point. Code points always end with a 0 high bit. i.e. From any byte move back until one finds a byte with a high bit of 0, that is the end of the previous code point; or one finds the start of the stream.
The point of self-synchronization is specifically that you wouldn't need to move back to know whether or not you're at the start of a code point.
> Also the requirements of charsets for storage and transmission are different for those in memory, ᴜᴛꜰ-8 is ubiquitous but web browsers must surely be ᴜᴛꜰ-16 to support Javascript.
What does this have to do with your new encoding?
gene@genethomas.com ║ https://genethomas.com