KerTeX: the engineering

RELEASE NOTES

2024-11-19: Compilation fixes for Plan9/9front. Plan9/9front APE sed(1) doesn't understand ranges and—at least under 9front and APE—printf(1) doesn't process the format. RISK has been modified to avoid these in order to work under Plan9/9front. Thanks to Elyes Kanzari for the problem report!


	2024-11-09: Compilation fixes for
newer versions of gcc. Newer version(s) of gcc is/are treating
as errors what were previously warnings about strict prototypes or
missing prototypes. The CWEB utilities were written in order to be as
portable as possible, including with old C compilers, and were
omitting full prototypes, as well as the use of void when no
parameters. This has been fixed, both in RISK—that may provide, on
demand, ctangle—as well as kerTeX. Thanks to Antonio Olivares for
reporting the problem! 
	
	2024-07-07 : Modification of
risk_comp to circumvent a bug in Debian dash(1) system shell.
This is only to fix for a compilation error due to a bug in Debian's
dash. There is no change for the resulting installed kertex_T.
Thanks to B. Atticus Grobe for report, tracing the offending code
and testing the solution!
	
	2024-05-21: Fix of a blunder in
the handling of third parties packages (recipes). The POSIX.2
library used for handling what is related to packages can be updated
separately from the core system, without rebuilding everything.
Unfortunately, a blunder made the "updated" version available for
download having the same release number as the good, in the
distribution, but with a blunder; and, when updating, this "same"
library overwrote the good one installed making recipes failing. It
has been corrected and a whole release has been done. Warning! One has to download new versions of
get_mk_install.sh or get_mk_install.rc!
The reason has nothing to do with the previous problem: I had
"upgraded" CWEB with a new version not anymore under Silvio Levy
responsability; unfortunately, this version depends on a LaTeX file
and, even more problematic, debugging of included cweb files doesn't
work anymore. So I have reverted to the last Knuth and Levy versions,
and there is not anymore a cweb tarball to unpack. Thanks to Romano
for making the report about the former problem on 9front!
	
	2024-05-21: LaTeX recipe
updated. The LaTeX recipe has been updated to match what is
current (including sources organisation) on CTAN as of today—the
kergis.com cache has been updated as well. Thanks
to Jack Lawler for reporting the error! 
	2024-02-16: Important informations
concerning the new recipes (for LaTeX users). This has to do
with the gestion of languages (Babel).
See Managing additions to the kernel.
	
	2024-02-14: Fix to recipe
pl@fonts.sh. The directory name in the zip file has changed and
the recipe has been thus fixed. Thanks to Antonio Olivares for the
error report!

	
2024-02-01: Fix to recipe
urw@fonts.sh. The Classico part was updated, and the list was
incorrect. Thanks to Antonio Olivares for the report!
	2024-01-29: Update to the
extensions framework. An important update to the extensions
framework has been made as well as updating the recipes for the
current state of CTAN. There is no need to reinstall the system, but
simply to update the pkg framework.
See Managing additions to the kernel.
	2023-08-01: kerTeX / MetaPost: fix
for a fault introduced when simplifying the search for an input'ed file.
When simplifying the search of an input'ed file, that was
"stuttering" in case of failure—the first search already searched for
all alternative, so retrying was yielding exactly the same result—I
had suppressed an alternative branch of the code, useless for the
result, but inside which an automatic variable was defined ensuring that
it was then defined also in case of failure. The result was that, in
case of failure to find the file, the automatic variable had a random
value, in all probability different from zero and was interpreted as
"true" while it should have been false. Coredump was following.
Corrected. Thanks to Dirk-Wilhelm Peters for reporting the problem! 
	
	2023-08-01: kerTeX / Prote: file
primitives behavior. The new file primitives are expected to
behave like the input primitive i.e. to add a ".tex"
extension if there is none. Prote modified to do so. Needed for new
LaTeX.
	
	2023-07-25: >kerTeX: CWEB update and
armv8l parm addition.
	The CWEB utilities have been updated to the 4.9 version now
maintained by Andreas Scherer, and support has been added for armv8l
(v8 ISA for 32 bits version in little endian).
	One has to use the new version of get_mk_install since there is the addition of a
separate sources tarball: the one for CWEB separately maintained.
	Thanks to Antonio Olivares for bug reports on Android!
	
	2023-05-09:Fix for latex.sh on Plan9.
	Sed(1) on Plan9/9front doesn't handle characters classes.
	Thanks to Jack Lawler for the report and fix!
	
	2023-05-08: get_mk_install.sh.
	The script now verifies that the user given (if distinct from the
	default) does exist—the installation does NOT create the user and
	group—and that he belongs to the group given.
	
	2023-03-17: LaTeX: recipe updated in
	order to allow to add various different languages hyphenation
patterns and exceptions, by Babel names.
	By defining on the command line (or in the environment) the variable
	SUPPORTED_LANGUAGES with a blank separated list of Babel
	names, the hyphenation patterns and exceptions of these languages
	are compiled in the format. The switch, afterwards, has to be done
	with Babel normal macros. Exemple:

 SUPPORTED_LANGUAGES="french ukenglish ngerman russian" $KERTEX_SHELL latex.sh install

	will add hyphenation for french, english UK variant (an extension of D.E.K.
	`hyphen.tex', new german (post 1996) and russian, the related
	"user level" Babel related packages having to be added afterwards,
	via their dedicated kerTeX recipe.
	
	2023-03-05: kerTeX_M: fix for
	compilation on RHEL. On RHEL, the order of the objects
	linked, at least when one is called "lib.o", does matter. Thanks
	to B. Atticus Grobe! for the report.
	
	2023-01-19: kerTeX 0.99.19.01:
Increase the memory (BIG) for current LaTeX on CTAN.
The current LaTeX version on CTAN
needs more memory in TeX (the memory is a compile time defined constant).
Thanks to Antonio Olivares for the report!
	
	2022-11-06: Fix for the compilation of
	kerTeX_T on Plan9.
	A macro definition related to the POSIX version is needed in order
	to compile some system dependent code (for the implementation of
	one of the supplementary routines required by LaTeX).
	Reported by Skip Tavakkolia.
	Also, fix of a blunder, spotted by Martin Ruckert, in
	input.ch (but that had no functionnal implication—no
	need to upgrade just for that). Thanks to both for reporting!
	
	2022-10-13: Fix to the LaTeX recipe for Plan9.
	Some commands added to the recipe recently were not matching Plan9
	usage (rmdir, mv directories...) and the call to mf was
	done without ensuring that the binary was in the PATH/path. Fixed.
	Thanks to Chris Gorman for the report!
	
	2022-09-26: Update of the LaTeX recipe
		. The recipe has been updated to add various files 
		needed to process at least the bbold@fonts.sh dtx when
		adding the support of these fonts for LaTeX (compatibility?).
		Thanks to Antonio Olivares for reporting and testing!
	
	2022-09-12: Update of the
	LaTeX recipe (2022-06-01-PL5). The recipe has been updated
in order to work with the current version on CTAN. Note: if the name of
the files have been unchanged, a recipe will work even with a more
recent version on CTAN. In case of a change, the recipe will fail and
has to be then used with the KerGIS cache of the files matching the time
when the recipe was written: $KERTEX_SHELL rcp.sh -p http
install. Thank you to Antonio Olivares for reporting the
incompatible change.
	
	2022-09-08: kerTeX minor release.
Support for FreeBSD fetch as HTTP client (it supports CTAN
CDN mirror redirections)—one has to use the new version of
get_mk_install.sh to select it at installation time). 
Also, improvement of mp2ps for isolated MetaPost figures.
	
	2022-05-18: Two new LaTeX packages:
exam and lcg. (Suggested for addition by Antonio Olivares...)
	
	2022-04-28: LaTeX package.
	Another bug fixing: the search path for fonts (installed
by this very recipe) was not specified; but the LaTeX ones are mandatory
for the format; hence they were searched with the default paths (the
kerTeX installation on the node) and found only when LaTeX has been
previously installed, and not found in a fresh install. Fixed. Thanks
to Antonio Olivares for the bug report!
	
	2022-04-26: Connection problems for
recipes (pkg). The recipe latex.sh uses a server
address that is in fact a CDN, redirecting to HTTPS servers depending on
load and location. Two problems : some http clients (depending on the
host system) are not able to handle the redirections or
(ftp(1) on FreeBSD) do not handle HTTPS; et, secondly,
 curl, that can handle both (but with some flags)
is rejected by some servers when identified as Curl... If the default
client does not manage to download, use for example lftp(1)
by declaring it this way:

sys_httpc=lftp sh latex.sh install

or update the pkg management in kerTeX by:

sh $KERTEX_LIBDIR/pkg/rcp/tools@pkg.sh install
that will allow (or should...) to use curl as
client (sys_httpc=curl). Finally, you can also replace in
the latex.sh recipe the server line:
http://mirrors.ctan.org/ by
http://downloads.kergis.com/kertex/pkg/src/ that has been
fed with a copy of the data required by latex.sh, and that
doesn't use a CDN nor require HTTPS (it serves HTTPS too, but it is not
a requirement).
	2022-04-21: latex.sh: bug fix; kerTeX
0.99.17.1: increase of BIG capacity for LaTeX. A blunder in the
latex.sh was not installing all the fonts related files
relative to psnfss. Fixed. Thanks also to Jack Lawler! for signaling a
portability blunder in the recipe (unzip used instead of
$PKG_UNZIP). Pour kerTeX: capacities had to be increased for
LaTeX documents defining a lot of control sequences. Also correction of
an error emitted when meeting an end of file without the line being
ended by a end of line. Ease of use: some documents referred to
epsf.tex as epsf.sty: links added. Thanks to
Antonio Olivares for the reports!
	
	2022-04-18: latex.sh: bug fixes
. Graphics was included but not mentionned in KXPATH; psnfss was
incompletely included, all the pfb definition fonts missing and the
dvips maps as well. Corrected.  Thanks to Antonio Olivares for the
report!
	2022-04-17: kerTeX 0.99.16.0: Prote
default engine and LaTeX3 compatibility. Release of the stable
version with Prote as default TeX engine (TeX, e-TeX compatibility;
additional primitives required by LaTeX3, and input handling compatible
with Web2C (TeXLive) version). Also: the curdir specification
"./some/file/path" leads to the pathname being taken as is, without
searching (the curdir is the process one; not the dirname of the first
input script file). The latex.sh recipe has been updated
to allow to compile and install the latest version on CTAN (LaTeX
related recipes—the present ones should be compatible—will be
updated progressively when time permits).
	2022-03-25 :Prote (devel):
implementation of supplementary methods for the input primitive.
Prote—default engine now for TeX, eTeX and LaTeX—now
accepts a double quoted argument (allowing spaces in the name) and
a grouped argument subjected to various expansions as a file name
definition for the input primitive.  Explanations can be found in
the input.pdf document on the dedicated Prote
page. Thanks to Phelype Oleinik for the test file about the expected
behavior of the implementation! (Note: the recipe for LaTeX is
unchanged, waiting the verification that the double quotes dance
does not hide any devil in a little corner.)

	2022-03-07: Prote: dev version
	0.99.15.9: allowing spaces in filenames at the general
	shared library level.  The majority of the engines
	used nowdays with LaTeX allow to specify as input, filenames
	with spaces (while, in the original code, spaces where
	delimiters and were thus forbidden in filenames). In order
	to achieve this, the shared library used for file searching
	and opening by all the utilities (whether written in
	WEB/Pascal or in C) had to be adapted and all the WEB/CWEB
	change files and/or the C code modified for the new API.
	The modification of the TeX input primitive will come next.
	This is an alpha release.

	2021-10-14: Prote has been successfully
	tested on NetBSD, FreeBSD, DragonFly and Android, on amd64, aarch64
	and earmv7 machines. As usual, thanks go to Antonio Olivares
	for being instrumental in testing on a lot of different systems!
	2021-10-11: Prote has been integrateg in the development
	version of kerTeX. This version is available for development
	by selecting dev when get_mk_install.sh (or
	get_mk_install.rc) asks for the version. Please see the
	Prote page on this site for more informations!
	2021-09-21: Prote: new engine, TeX and
	e-TeX compatible, with extensions. In the same spirit as e-TeX,
	Prote is developed on TeX and e-TeX, being compatible with these and
	adding extensions. The first extensions are the latest LaTeX
	required primitives (missing in TeX and e-TeX). The engine will be
	incorporated soon in the kerTeX distribution.
	2021-09-02: Security fix for
	TeX, METAFONT, MetaPost and e-TeX. A bug, in the handling of
	the filenames in extrem cases has been found, and the present
	correction at least ensures that when writing the log or dvi files,
	the extensions ".log" and ".dvi" are always present. If you have
	installed the LaTeX recipe, after upgrading, please re-install
	LaTeX so that the underlying engine is the updated one (e-TeX).
	2021-06-27 : Fix for kerTeX_T:mp2ps. echo
	treats or not escaped characters (this is implementation defined
	in POSIX.2). With the Pi shell, escaped characters are interpreted
	leading to an incorrect generation of the TeX template. This is
	corrected—this is the only modification and if mp2ps is
	working on your system, there is no need to update. Thanks
	to Antonio Olivares for reporting!
	
	2021-04-22: D.E. Knuth's 2021 tune-up incorporated.
	D.E. Knuth has revised the TeX, METAFONT and al. system in early
	2021. KerTeX has been updated to latest versions, e-TeX and 
	MetaPost being adapted to match also.
	
	2021-04-07: Fixes for some babel recipes (pkg). The
	Babel's recipes belarusian, friulan, georgian, hungarian, italian,
	occitan, piedmontese, romansh and slovenian have been fixed for
	a harmless bug: there is no .ins generated so the processing
	of these shall not be called. Thanks to Antonio Olivares for the
	report!
	
	2021-03-08: fixes for Plan9. Plan9 doesn't provide
	rmdir and mv renames/moves files but not
	directories. Fix in RISK, RISK version adjustement in kertex_M,
	fix for mp2ps in kertex_T, and fixes for the recipes
	(pkg) latex.sh and amstex.sh. Thanks to
	Gilman Ironworks for reporting!
	
	2021-02-11: fix of RISK for Raspberry Pi OS. The
	dynamically shared libc needs a second static library chunk whose
	informations are given in a pseudo libc in /usr/lib/... that is
	actually a ld script.
	Searching first /lib/... finds the true libc file but then, when
	linking, some symbols are not resolved. Solution: modify the order
	of the search for raspbian. (Thanks to Antonio Olivares for the
	report of the failure with the new version of Raspberry Pi!)
	
	2020-11-22: fix to latex.sh recipe. The new recipe now
	integrates required packages (ams, babel core, cyrillic, graphics,
	tools—and ec fonts) that were separate packages before. Hence,
	after installing the new version, the latex recipe automatically
	removes the obsolete packages it replaces. But the new files were
	installed in the very same directory as the resp. old package.
	Hence, it the old package was installed, removing it removed the new
	files that have overwritten the old... Fixed. Thanks to Antonio
	Olivares!

	2020-11-22: RISK and kerTeX_T fixes.
RISK fix for untested "cd try_dir"
that might fail (Linux); RISK fix and kerTeX_T fix for static
compilation under Linux based distributions (tested on Android). Thanks
once more to Antonio Olivares for the reports!
	

	2020-11-18: RISK fix for FreeBSD and russian
and ukrainian recipes fixes. RISK: Search for clang
and do not take c99 that accepts only POSIX options. Russian
and ukrainian: batchfile definition for docstrip has to be
kept as is. Thanks to Antonio Olivares for the reports!
	

	2020-11-17: RISK fixes for OpenBSD and
arm64. Fixes have been applied for compilation on OpenBSD;
paramaters file has been added for arm64 (aarch64). Thanks to Asad
Lodhia for reports and tests!
	

	For LaTeX (babel) language recipes. The autogeneration
	generation of files is done via docstrip, that is provided by LaTeX
	and hence has to be done by latex and not
	tex—docstrip is accessible by tex if and only if
	babel@tex has been added. Thanks to Antonio Olivares for reporting
	the error!
	

	carlisle@latex recipe added for miscellaneous style
	or macros files written by David Carlisle (scalefnt.sty for
	example).


Git sources are available here:


	The RISK compilation and installation
framework
	WEB to C translation
	the kernel
	Recipes currently written for extensions (LaTeX
etc.)



The current published version is
stable: 0.99.23.00 (CWEB update). (No new functionnalities:
fixes for mp2ps for Pi.)


Rules about versioning

The version identifier is, rather classically, composed of 4 dot
separated numbers:
major.minor.revision.patch_level.

When the minor is 99, this means that we are aiming to
major+1; i.e. n.99.x.x means with are aiming
to release the n+1 major version.

When both minor and revision are 99, this is an alpha of the
major+1, the path level number in this case can be
incremented from time to time when releasing new alphas versions.

When the version is major.99.99.99, this is a beta
release, i.e. the last publication to verify that everything is correct
and to have the opportunity to iron out the last frequent unfelicities
before releasing major+1.

News: support of the present version

As indicated by the version, we are aiming to release major 1, but
some steps have to be accomplished before:


	 OGO display (METAFONT) for rio on Plan9;
	MS Windows native support with Msys2 (this is not kerTeX
	by itself but the RISK framework);
	OGO display (METAFONT) for GDI on MS Windows;
	Improving of the administration of the pkg framework (see 
	below);
	Extension of ε-TeX to implement
	LaTeX newly required primitives starting with version
	2020-10-01-PL2,
	adding to the ε-TeX primitives, primitives that are found
	in PDF engines—but we won't implement PDF bound output primitives,
	only primitives that are not linked to the output format; we will
	keep DVI and even extend it finally with graphical functions, but
	later. We will try to include too the micro-typography primitives
	present in PDFTeX. (for the moment inessential: the engine will be
	called reTeX, ζTeX—the letter following
	ε—or kTeX— pour KerTeX.)


Since the base of the system (what is precisely relative to fonts
present in base or to the two engines: TeX and &epsilon-TeX) is very
stable, and in order to be able to use the main spare and sparsed time
that can be dedicated to kerTeX on the bulk of what has to be done,
changes to the present version will be restricted to corrections of
problems arising during installation on systems (RISK), or for adding
new systems (native MS Windows excepted for the moment), but we will
not add new recipes (pkg) to the ones already present and updated
to the end of october 2020 (between 26 et 28) versions on CTAN. For
LaTeX, the version is "frozen" for now, for reasons explained above.

For the packages infrastructure, that has an OS dependent part,
it is now possible to update it, without recompiling everything, by
using the very packaging system itself: the recipe
tools@pkg.sh allows the installation of an update; and 
rcp@pkg.sh allows to download the latest version of all the
existing recipes. 

User level: the recipes (rcp) and the packages

We finally gave an name to the script chunks defining the
instructions to download, put in form and install additions to the
base system and we have added functionnalities to dissociate this user
level stuff from the kernel.

Practice has proven that the basic principles of the packaging framework
work well. But there are some unfelicities when it comes to
administration, and it would be for example better if the system
would recursively be able to create packages a given package depends
upon, without requiring to install these dependencies first (avoiding
the need to switch to a privileged user when the installation is a
privileged one). These points will be solved for the 1.0 release.

As mentioned above, LaTeX—that is not an engine but a TeX macros
framework—could not be installed anymore, some months ago, on the TeX
engine, requiring the ε-TeX extensions, and we had to modify
the latex.sh recipe to adapt (it was possible because
kerTeX provides ε-TeX); but now, LaTeX requires primitives
that are supplementary to these ones and that are found in the PDF
engines. Hence LaTeX is for now frozen to the february 2020 release.
This should not be of a great concern for LaTeX users since this is,
at least, the more up-to-date version on the other TeX distributions and
it should be months before an end user will see a difference between the
"new" version and the february 2020 one.

The recipes system had this advantage that, in the way it has been
engineered, the recipe was in the majority of cases still valid for
a new released version on CTAN as long as the processing of the 
generation of the files did not change and the requirements about the
engine were the same. It was indeed a TeX "live", the user downloading
the latest CTAN version.

But this advantage was, de facto, also a drawback: if some
incompatible change was made upstream, the recipe failed. In the absence
of a versionning system, the package was not installed until someone
updated the recipe.

With the major modification of LaTeX prerequisites, we had to adopt a
more safe way (while still keeping the ability to test live updates...).
Now, the CTAN sources of the "working" version of the recipes is
cached on KerGIS.com http server.

More long term aims

There are some not to say quite some. But in order to be able to
dedicate the majority of a restricted time to address what has been
exposed above, I will not discuss them for now (there are quite some
points, and I'm inclined to think that they are quite entertaining).
First things first: the 1.0 release...


Thierry LARONDE

tlaronde @ polynum.com

(please use your e-mail client with the above address)