Hyperlinks broken because of subfunctions

[ecm-pushbx]

Working with intlist.pl I found the expected broken links in the INT 21 section: 21.71, 21.30, 21.33, 21.4B, 21.58 subfunctions. These are variously referenced eg INT 21/AX=4B01h which leads nowhere as there's a single entry for 21.4B that includes AL values including 01h:

TheList/source/Interrupt List/INT 21 DOS Function Calls/INT 214B DOS 2 EXEC LOAD ANDOR EXECUTE PROGRAM.txt

Lines 8 to 13 in 8a9c399

	AH = 4Bh
	AL = type of load
	00h load and execute
	01h load but do not execute
	03h load overlay (see #01591)
	04h load and execute in background (European MS-DOS 4.0 only)

21.6C also is sometimes referenced without the full AX value. Some links specify a "subfunction" using the syntax for a nonexistent register "SF=" (these are not picked up by intlist.pl currently).

And 21.440D has a bunch of links specifying a CX= value. It seems these fail because the line CX = 086Bh / 486Bh isn't picked up by my script as a valid register statement:

TheList/source/Interrupt List/INT 21 DOS Function Calls/INT 21440DCX086B MSDOS 70 GENERIC IOCTL UNLOCK PHYSICAL VOLUME.txt

Line 9 in 8a9c399

CX = 086Bh / 486Bh

[ecm-pushbx]

I fixed the 440Dh subfunction hyperlinking, by allowing register statements of the form CX = 086Bh / 486Bh with two or more slash-separated values, which enter both/all values into the summary (comma-separated) and prior registers match value (which is an array now to allow multiple values): https://hg.pushbx.org/ecm/tractest/rev/293321c033ea

[ecm-pushbx]

The slash-separated list could be used for 21.4B as well. Maybe like so:

  AX = 4B00h / 4B01h / 4B03h

Or maybe like so:

  AH = 4Bh
  AL = 00h / 01h / 03h

The latter would require some logic in the registers match to allow an INT 21/AX=4B01h hyperlink to match this entry, but that is possible.

[ecm-pushbx]

We could also try to parse this (as is) register list:

 AL = type of load
     00h load and execute
     01h load but do not execute
     03h load overlay (see #01591)
     04h load and execute in background (European MS-DOS 4.0 only)

But how to match this without any false positives? The whitespace would probably need to be significant to match this.

Another example:

TheList/source/Interrupt List/INT 21 DOS Function Calls/INT 2158 DOS 5 GET OR SET UMB LINK STATE.txt

Lines 8 to 17 in 8a9c399

	AH = 58h
	AL = subfunction
	02h get UMB link state
	Return: AL = current link state
	00h UMBs not part of DOS memory chain
	01h UMBs in DOS memory chain
	03h set UMB link state
	BX = new link state
	0000h remove UMBs from DOS memory chain
	0001h add UMBs to DOS memory chain

[shidel]

@ecm-pushbx, this and similar reasons is why I decided trying to use fix-ids was not worth the effort. While the algorithms to process those things was getting pretty good. It did not result in any more uniqueness. Just, the need to exclude a register like AL because it was used many times with multiple values.

Generally, the different levels of whitespace sort of did the trick when combined with additional tests. Things like SPACE4 + REG [SPACE]= [SPACE] HEXVALUE + [SPACE] + IGNORED and not (INDENTED)[SOMETHING]:, and other such tests. With the eventual [SOMETHING]: stop processing header. Something being a group of letters without any spaces like Return:, SeeAlso, etc.

I am just starting to dive into the new makehtml. At present, it is at the stage where it will separate the files and their data into chunks. After that will come extracting the usable information to use for creating entry cross-linking. I will definitely try and keep you posted on my findings on these issues.

A couple things of note I found while working on the abandoned fix-ids regarding registers in the Interrupts.

• There were no entries found where there was a conflict between 16 and 8 bit registers. In other words, it didn't see anything like BX = 1234h and before or after have `BL = 67h'.
• Only one entry, used CL and a hex value in the unique ID, but also in the INT had a CH=hex value. So, it could have (or maybe, should of) actually had a CX???? for its unique ID. But, a related INT only had CL=?? and no CH declared. So, I ignored that.
• Those SF (SubFunction) ones are a little annoying. But, generally you can just treat them like another register.

However, I do not like using SF. My gray matter wants to think Sign Flag. Maybe, we should rename those to FN or something to prevent confusion. Or maybe, just leave them be and if a parser like makehtml_ wants to provide some hover text for things like ZF, CF, SF, etc. it will just need to be smart enough not to do that inside any cross linking references.

:-)

[ecm-pushbx]

@ecm-pushbx, this and similar reasons is why I decided trying to use fix-ids was not worth the effort. While the algorithms to process those things was getting pretty good. It did not result in any more uniqueness. Just, the need to exclude a register like AL because it was used many times with multiple values.

Generally, the different levels of whitespace sort of did the trick when combined with additional tests. Things like SPACE4 + REG [SPACE]= [SPACE] HEXVALUE + [SPACE] + IGNORED and not (INDENTED)[SOMETHING]:, and other such tests. With the eventual [SOMETHING]: stop processing header. Something being a group of letters without any spaces like Return:, SeeAlso, etc.

I'm doing something similar in the readline_replace function where it reads the first line after a -------- separator as the base summary and builds up register values from tab-indented lines after the summary base line. It does require that the register statement lines only contain register values, no "ignored" parts afterwards. Register value ranges (with .., to, or -) are currently recorded like just the first value. Single values may be followed by quoted text that matches the hexadecimal value, either in NASM order (which matches the order of words read from memory) or MASM order (the reverse, which matches "most significant byte first" intuition). And as mentioned, multiple slash-separated are allowed now too.

Hyperlinks with register values specified match the first entry whose summary matches the interrupt number and all register values (if any) given in the hyperlink, as well as the "name" part (in double quotemarks).

I am just starting to dive into the new makehtml. At present, it is at the stage where it will separate the files and their data into chunks. After that will come extracting the usable information to use for creating entry cross-linking. I will definitely try and keep you posted on my findings on these issues.

A couple things of note I found while working on the abandoned fix-ids regarding registers in the Interrupts.

* There were no entries found where there was a conflict between 16 and 8 bit registers. In other words, it didn't see anything like `BX = 1234h` and before or after have `BL = 67h'.

That's good to know.

* Only one entry, used CL and a hex value in the unique ID, but also in the INT had a CH=hex value. So, it could have (or maybe, should of) actually had a CX???? for its unique ID. But, a related INT only had CL=?? and no CH declared. So, I ignored that.

Yeah, the lower/higher half registers might have to be matched specifically if you get a link like INT 21/AX=4B01h but the entry has AL=01h and AH=4Bh.

* Those SF (SubFunction) ones are a little annoying. But, generally you can just treat them like another register.

Are there any entries where a tab-indented line actually reads SF = 1234h though? If not then treating SF= as a register just means the hyperlink is treated as broken.

However, I do not like using SF. My gray matter wants to think Sign Flag. Maybe, we should rename those to FN or something to prevent confusion. Or maybe, just leave them be and if a parser like makehtml_ wants to provide some hover text for things like ZF, CF, SF, etc. it will just need to be smart enough not to do that inside any cross linking references.

:-)

I don't hate it, SF only pops up in this case where registers are expected.

[ecm-pushbx]

IntList in https://hg.pushbx.org/ecm/tractest/rev/9b11a45e8c36 now detects lines starting with <TAB>AL = subfunction (and two more specific alternative texts) followed by <TAB><BLANKx4><HEX> as a subregister, until the next <TAB><NONWHITESPACE> (ends the subregister search) or <NONTAB> (ends the entire summary buildup). Lines with two tabs or more than tab + 4 blanks indentation, or where there isn't a hex value directly after the tab + 4 blanks, are skipped.

Furthermore, in https://hg.pushbx.org/ecm/tractest/rev/77dc3c45c3a8 if a summary is detected with (at least one) AX= value but without AL= and AH= then artificial AL and AH entries are added to the register match hash. And if there's AL and AH but no AX, the reverse operation is done. This allows linking to 21.6C as INT 21/AH=6Ch despite the summary listing AX, and 21.4B as INT 21/AX=4B01h despite the summary listing AH=4Bh/AL=00h,01h,03h