2997lc-0002lc-0002.xmlForester for the Woodland Skeptic2024218Owen LynchtrueWhat I don’t understand about Forest (and what is keeping me from just go ahead and write everything there) is the syntax… why a new one? Why not Markdown or LaTeX or a reasonable hybrid like the nLab? -- Matteo CapucciThis is a tutorial aimed at people who want to get started writing on the LocalCharts forest, but don't know how forester works/are frustrated at how forester is different from tools they may be used to. This is intended to be a direct introduction than the official tutorial (which has more motivation for forester), and also has some localcharts-specific content.2775lc-0003lc-0003.xmlForester in 30 secondsForester is a tool for writing large quantities of professional quality mathematics on the web. To support large quantities, it has extensive interlinking/backlink/transclusion support. To support professional quality, it has a uniform, LaTeX-like syntax with macros that work the same in prose, KaTeX-based math equations, and LaTeX-produced figures.An example forester document showcasing basic features looks like this:and would be rendered like this:2776lc-0006lc-0006.xmlExample document197011Owen LynchA paragraph with inline math: , and then display mathbold and italicsa bulleted lista numbered list Called my-macro with argument tweedle2779lc-0005lc-0005.xmlA subsectionMore stuffForester is written in .tree files in the trees/ directory. Trees are named namespace-XXXX, where XXXX is a number in base 36 (using digits 0-9 then A-Z), and namespace is typically something like the initials of the person writing.Now that you've seen the overview, I encourage you to try writing a page. Instructions for setting up the localcharts forest are in the README for the localcharts forest. Once you've done that, you can come back here and learn forester in some more depth.But before you go on, four things.There is a convenient script for making new trees numbered in base-36: new <namespace>. For instance new ocl produces a new file trees/ocl-XXXX.tree, where XXXX is the least strict upper bound of the set of YYYY such that ocl-YYYY exists in the forest. We use this for most prose, but we also use the authors/, institutions/, etc. directories to put bibliographic trees, named after their contents instead of automatically named. If you have any questions or complaints about this tutorial, please comment below. If you have general questions about forester, I encourage you to join the mailing list. Don't be afraid to poke around at either the source code for localcharts or the source code for Jon Sterling's webpage; you may find some interesting features! Be aware that forester is new software in active development, and Jon Sterling is not afraid to make breaking changes. This is a blessing and a curse; he might write something which breaks things, but he also might write the feature that you really want if you ask nicely on the mailing list!2780lc-0004lc-0004.xmlBasic TypographyOwen LynchForester has a lot of interesting features around large-scale organization of documents, but it's worth reviewing the nuts and bolts of basic typographical constructions first, so that you have tools in your toolbox for when we move to the large-scale stuff.Forester's syntax is a little bit unfamiliar, however I hope by the end of this section, you will see that the design decisions leading to this were not unreasonable, and lead to a quite usable and simple language. This is because there is mainly just one syntactic construct in forester: the command. The syntax is , which should be familiar to anyone who has used LaTeX.2782lc-0009lc-0009.xmlHTML TagsThe most basic commands just produce html tags, as can be found in the table below. Name HTML Forester Paragraph ...

]]> Unordered list

...

...

]]> Ordered list

...

...

]]> Emphasis ...]]> Strong ...]]> Code ...]]> Pre ...]]> Blockquote ...]]> Note that unlike in markdown or LaTeX, paragraphs must be explicitly designated via , rather than being implicit from newlines.Also note the conspicuous absence of the anchor tag: we will cover links in .It is also possible to output any other HTML tag via the command. For instance, I used to produce the table above.So to sum up, basic typography in forester is just a LaTeX-flavored wrapper around HTML elements. Given that we have to use LaTeX for math (because it's too much effort to relearn how to typeset math euations), it makes sense to just have a single syntax throughout the whole document.2820lc-000Alc-000A.xmlMath TypesettingTo produce inline mathematics, use . Math mode uses KaTeX, so anything supported in KaTeX will work in forester. Note that math mode is idempotent: produces the same result as . Display mathematics can be made with .The killer feature of forester is the ability to include LaTeX figures as svgs. This is done by compiling the figure with the standalone package, and then running dvisvgm. The results are cached, so that when you make changes elsewhere in your document, the figures do not have to be recompiled.You can access this feature with the command. The first argument is the preamble, where you can put your statements. The second argument is the code for your figure, using TikZ or similar packages. The localcharts macro package (accessed with ) also provides a command which has the right preamble for copy-pasting quiver diagrams. Note that you must remove the surrounding from the quiver export, or you will get weird LaTeX errors.2821lc-0007lc-0007.xmlReferences and TransclusionsOwen LynchIn this section, we discuss how to handle ideas that are not contained in a single file in quiver.2823lc-000Blc-000B.xmlLinksThe easiest way to connect ideas is via links! Forester supports several different types of links. The simplest is markdown-style links, written like . Because linking to the nlab is so common in localcharts, we also have a special macro (when you have at the top of your file) for linking to nlab pages by their title, tastefully colored in that special nlab green like so: Double category.Additionally, pages within the same forest can be referenced just by their tag, as in , or "wikilink style" with , which produces a link titled by the title of the referred page, like so: LocalCharts Forest. Note that internal links have a dotted underline. Moreover, on a given page , one can see all of the other pages that point to via internal links.2825lc-000Clc-000C.xmlTransclusionTransclusion includes the content of one file into another file. The basic command for transcludes is . This is similar to the LaTeX support for multi-file projects, but is used much more pervasively in forester. For instance, instead of a command, the general practice is to make each section a separate .tree file, and have the larger document them.This is also how mathematical environments are supported. What in LaTeX would be something like:in forester is generally accomplished by creating separate .tree file with at the top. When transcluded, this looks like a definition environment. For example:2826ocl-0015ocl-0015.xmlDiscrete Interval CategoryDefinition202421The discrete interval category is the category where objects are sets , and a morphism from to is a choice such that . In other words, it is a map that preserves the linear order and metric structure.In general, is used to designate the type of a tree, like definition, lemma, theorem, etc., but also non-standard ones like person, institute, reference. You can search for taxon in the LocalCharts forest to see how it is used.Splitting up your writing between so many files can be a pain, but there is a reason behind it. The philosophy behind forester is that in order to pedagogically present mathematics, it is necessary to order the web of linked concepts in some logical manner. However, this order is non-canonical. Therefore, we should support multiple orderings.From a more practical perspective, one gets tired at a certain point of clearing one's throat in the same manner every time one starts a new paper or blog post, reviewing similar definitions to get the reader up to speed. Being able to reuse parts of a document can alleviate this.With transcludes, there is also yet another type of linking: a cleveref style command, which produces references like when the referenced item is contained within the current tree, and when the referenced item is not contained within the current tree.2827lc-000Dlc-000D.xmlBibliographiesBibliographic citations are just trees with at the top. See here for examples of how to write these. Whenever a reference is linked to in a tree, the bibliographic information appears at the bottom of the page. For instance, I can link to Topo-logie and the formatted citation should appear at the bottom of this page.2828lc-0008lc-0008.xmlMacrosOwen LynchOne of the main reasons that I chose forester of all the ways of writing math on the web was its support for macros. It pains the soul to have to write all the time; I just want to write !Macro definition is similar to LaTeX, but with two changes. The first change is that it uses instead of . Recall that was the original syntax in TeX, but had some warts, so LaTeX had to change to , so this is really just going back to the roots of TeX. The second change is that arguments are named instead of numbered. For instance, instead ofyou would useYou can take a look at macros.tree for the LocalCharts "macro standard library". To use this in one of your trees, you must have at the top. Finally, note that the same macro definition works in prose, within or , and also within , so you can use the same math abbreviations in your inline equations and in your commutative diagrams, just like you would in a real LaTeX document.2830lc-000Elc-000E.xmlFrontmatterYou may have noticed that there are certain commands that tend to go at the top of documents, like , , etc. These are the frontmatter of the document, and provide metadata for the content.Most frontmatter commands can be found here.In this section, we document some other useful frontmatter commands not covered in the above.One important command for bibliographical trees is . This is used for many sorts of metadata, like doi, institution (used on author pages), external (used to provide an external url, such as a link to a pdf). Particularly important for localcharts is , which enables the discourse integration on a particular tree so that the tree is automatically crossposted the first time it is visited, and comments from the forum show up beneath it.Another useful command is . This command is used to add "tags" to a page which might give some hints as to the subject matter. Tags are not displayed on the page, but pages can be queried by tag. This is very useful for producing bibliographies of references on a certain subject, as I did here.That's all for now; it's time for you to go forth and write some math!References2999anel-joyal-topo-logie-2021anel-joyal-topo-logie-2021.xmlTopo-logieReferenceMathieu AnelAndré Joyal10.1017/9781108854429.007 3002#947unstable-947.xmlNotes.Mathieu AnelAndré Joyalanel-joyal-topo-logie-2021 Great translation table between logic and logoi. (pg. 9) The Sierpinski locale is the free frame on one generator. (pg. 13) The poset of open subsets of a closed set , where , is the coslice of under : open subsets of are in canonical bijection with open subsets of containing . (pg. 15) Backlinks3005aria-0001aria-0001.xmlFormal and Informal Collaboration202434Owen LynchTopos InstituteA presentation at Davidad's second ARIA workshoptrue2591#249unstable-249.xmlOverviewSlide202434Owen Lyncharia-0001 Follow along at forest.localcharts.org/aria-0001.xml! Motivation Informal collaboration Formal collaboration 2593#248unstable-248.xmlNotes202434Owen Lyncharia-0001false This is the source document for my talk at Davidad's second ARIA workshop in Birmingham on safe-by-design AI. From this document, I produce the pdf presentation slides and this HTML forester page. 2595#251unstable-251.xmlMotivation: What does success mean?Slide202434Owen Lyncharia-0001 Successful implementation of Davidad's program thesis over the next 3 years implies something like Several new fields worth of novel math research ]]>1,000,000 lines of code This is only possible if we either: Clone Urs and ekmett a couple of times and form them into an (aligned) borg-like mindmass, or Get really good at collaboration. 2599#250unstable-250.xmlNotes202434Owen Lyncharia-0001false The code also needs to be correct and efficient. 2601#253unstable-253.xmlMotivation: Current de-facto standards for collaborationSlide202434Owen Lyncharia-0001 Informal technical writing: overleaf+arXiv Technical computing: github repositories containing arbitrary code 2603#252unstable-252.xmlNotes202434Owen Lyncharia-0001false If you are really lucky, the github repository will be a maintained package installable through a package manager, and if you are really really lucky, the maintainer won't graduate in a year and forget about it. 2605#255unstable-255.xmlInformal collaboration: optionsHiddenslide202434Owen Lyncharia-0001 Papers: too slow, standalone. Blog posts: better as advertisements/summaries for a larger audience. Also usually fairly standalone. Slack/zulip: too fragmented, too short Roam Research: not designed for real book-length mathematics. nLab: past attempts to encourage people other than Urs to do novel math on the nLab have failed, unclear why Gerby (stacks project software): powers arguably one of the most successful giant mathematical collaborations in history. Not explorative, janky Forester... just might work?? 2608#254unstable-254.xmlNotes202434Owen Lyncharia-0001false I've thought about this a lot: see a more complete survey here. 2610#257unstable-257.xmlInformal collaboration: A dreamSlide202434Owen Lyncharia-0001 Monday morning (UK time): DJM writes down new definition Monday afternoon (EU time): Matteo adds some key lemmas Monday afternoon (Pacific time): Sophie spots hole Tuesday morning (EU time): A long-time lurker comments for the first time on idea for patching hole ... By Thursday night: Enough material for a paper Friday morning (UK time): Organize material into a paper by writing an abstract, collating background+new material into a reasonable order, and then exporting to arXiv-compatible LaTeX. All authors of transcluded material notified and given a chance to review. Friday afternoon (UK time): pub. 2613#256unstable-256.xmlNotes202434Owen Lyncharia-0001false AND THEN WE DO IT AGAIN THE NEXT WEEK. 2615#259unstable-259.xmlInformal collaboration: How does forester work?Slide202434Owen Lyncharia-0001 Forester takes a collection of files with TeX-like syntax and produces both a static website and LaTeX. Key features: Transclusion Linking, backlinking, and citation Macros TikZSVG Customizable LaTeX export Better error messages than LaTeX 2617#258unstable-258.xmlNotes202434Owen Lyncharia-0001false Forester for the Woodland Skeptic is an introduction to forester for localcharts, and the official introduction is Build your own Stacks Project in 10 minutes. You can see the README Transclusion: the operad of writing Linking, backlinking, and citation: the best organization method known to humanity Macros: separation of intent from style TikZSVG: diagrams, diagrams everywhere Customizable LaTeX export: the magic of XML Better error messages than LaTeX Definitions, theorems, sections, references, etc. are all separate units (called trees) in Forester which can be freely included into other documents via transclusion, possibly recursively. In addition to transclusion, you can also just link other pages, which will automatically show up as a link to, say, Definition 3.4 if it happens to be on the same page, or Definition [double-category] if not. Each tree records which other trees link to it, and displays those trees at the bottom. Forester mimics the LaTeX macro system, and expands a single macro definition within text, inline math (which is displayed via KaTeX), and TikZ that gets compiled to svg. Macros also are tree-local rather than global to the whole forest, though one tree can import the macros from another tree. Specifically, you can just copy-paste commutative diagrams from quiver into forester. 2619#261unstable-261.xmlInformal collaboration: Just add (more) usersSlide202434Owen Lyncharia-0001 LocalCharts is live! Medium-sized forum Small but growing forester instance This talk built via forester Compatible with UK law for government projects 2621#260unstable-260.xmlNotes202434Owen Lyncharia-0001false I have spent the last year (actually funnily enough starting with the first time I met Davidad) setting up the localcharts ecosystem, which includes a collaborative markdown editor, a discourse server, and an automated build system for forester which builds the localcharts forest from its git repository. We have docs on how to use the localcharts forest. And the whole system runs on (EU/UK)-hosted services, fit for use in a program sponsored by the UK government. 2623#263unstable-263.xmlFormal collaboration: Math on the computerSlide202434Owen Lyncharia-0001 What does it mean to do math on the computer? Logician: propositions as types. Characteristic Algorithms: Martin-Lof type checking Programming languages: Isabelle, Lean, Coq, Agda Algebraist: Symbolic rewriting Characteristic Algorithms: Groebner bases, e-graphs Programming languages: Mathematica, Macaulay2, OBJ3, Z3 Engineer/statistician: Numerical computing Characteristic Algorithms: Euler's method, MCMC, gradient descent Programming languages: Fortran, MATLAB, Julia 2625#262unstable-262.xmlNotes202434Owen Lyncharia-0001false We are going to be doing math on the computer a lot in this program. But what does that actually mean? To a logician, that means turning propositions into types, and proofs into terms of those types, in a language like Agda, Coq, Lean, etc. In this way of doing math, however, the only algorithm is the type-checking algorithm. Of course, by the Curry-Howard correspondence constructive proofs are equivalent to functions, and those functions might be interesting algorithms. But there is a significant difference between the use cases of formal verification of an algorithm and proof by typechecking. It's nice to formally verify your algorithms, but when your task is write an algorithm for a task, the constraint that your program has to also be a proof of that program's correctness slows things down. And there are many algorithms other than the typechecking algorithm that we care about. For instance... To an algebraist, doing math on a computer means using a variety of algorithms to rewrite symbolic equations. One could see this as tactics for proof, but I have yet to see a Groebner base tactic or an e-graph tactic implemented in a proof assistant. It could be an interesting research task to build such a proof assistant, but this is somewhat of a distraction when prototyping new symbolic algorithms. And additionally, the task of writing a tactic engine is significantly enough different from writing proofs using that tactic engine that it's worth thinking about as a different way of doing math on the computer. Classically computer algebra is often done in in dialects of LISP, or special-purpose solvers written in high-performance languages like C++; we might want to do it in Rust or Julia. But that is not the extent of math on a computer. To an analyst, math on a computer means numerical methods. In the glorious future, we will have numerical methods written in Lean that compile to the GPU. But we are not yet in the glorious future. Especially for experimentation, we will want to use languages suited for high-performance which classically meant Fortran or C, but now includes Julia, Rust, the ecosystem of prewritten Fortran/C/C++ that has python wrappers, and Haskell DSLs that compile down to Fortran/C/C++/CUDA (which is how Ed Kmett 100xed state of the art performance for ray tracers). And then there is another task that requires a different set of tools: the task of having a UI that is not a github of script files, which is the current state of the art for scientific computing. Now, a github of script files is actually a pretty excellent UI; I'd take a github of script files over an opaque, half-baked web UI any day of the week. But if we want something like a database of models, then we may end up with UI that is not just writing scripts and putting them in git. All of these types of doing math on the computer are important for the program. 2645#265unstable-265.xmlFormal Collaboration: Polyglot Scientific ModelsSlide202434Owen Lyncharia-0001 Dilemna: Don't want to rewrite tensorflow Don't want to write everything in Python Solution: Models should be language-independent initial algebras of systems doctrines. Algorithms are model semantics that apply over large classes of models. 2649#264unstable-264.xmlNotes202434Owen Lyncharia-0001false We don't want to rewrite tensorflow, but we also don't want to write everything in python. Eventually, we should just develop a language which does it all. But we have to figure out what it all consists of first. And engineering is research; even though the final product might be a clean reimplementation of research prototypes, it's still important to surface engineering problems in research prototypes. These research prototypes by necessity will be in extant programming languages, using extant libraries, and there is not a single choice of extant programming language which completely covers all of the ways of doing math on a computer. In any case, we should be learning from the technological state of the art in different domains, and this necessitates multiple programming languages. Even if we manage to build the next great mathematical programming language, it will still be good to have language-independent models, because we aren't going to convince the entire world to use a single programming language, but we want to have safe AI for the entire world. Fortunately, we have Categorical Systems Theory, which tells us that a model is an initial algebra in some doctrine of systems. Elements of this initial algebra are just data that is independent of any particular computational paradigm. Of course, in order to implement all the operations of categorical systems theory, we will need to write functions for editing, composition, simplification, analysis, simulation, etc. These will operate on language-independent data, but will not themselves be language-independent data. Ideally, we write basic operations like editing, composition, simplification, etc. in a single language (Rust perhaps), and then offer some kind of ffi. 2654#267unstable-267.xmlFormal Collaboration: Models as DataSlide202434Owen Lyncharia-0001 What can be cross-language? Algebraic Data Types Generic types Multidimensional Arrays Presentations of algebraic structures (i.e. ring presentations by generator and relations). Knowledge bases, i.e. collections of facts in the style of prolog. What can't? Arbitrary functions Arbitrary dependent types 2657#266unstable-266.xmlNotes202434Owen Lyncharia-0001false Given that we want to use multiple languages, how can we nonetheless build off each other's work? One way is remote procedure calls. But there's a hard limit to what remote procedure calls can convey cross-language: that limit is the expressivity of the serialization format. You can't just serialize a function in a probabilistic programming language and ship it off somewhere. If you want to produce a scientific model in one language and then run analysis tasks on it with other languages, you need to be able to seamlessly serialize complex scientific models. So we need to carefully choose an integration strategy based on what is reasonable to serialize, that encompasses types complex enough to convey scientific models. 2660#269unstable-269.xmlFormal Collaboration: Implementation of Models as DataSlide202434Owen Lyncharia-0001 Type theory for data Embed into existing languages Build storage system 2663#268unstable-268.xmlNotes202434Owen Lyncharia-0001false Make a type theory expressive enough for the models we want, and such that all well-formed types make sense to serialize cross-language. I have some clever ideas for how to do this that I won't go into right now, but can be found (links) Write a compiler that compiles type definitions to produce wrapper types in all relevant languages, similar to protobuf. Build systems for storing serialized models that can be accessed programmatically. It would look something like: struct Graph { V: fintype; E: fintype; src: E -> V; tgt: E -> V; }; 2666#271unstable-271.xmlFormal Collaboration: Version the Source of Truth, Cache Everything ElseSlide202434Owen Lyncharia-0001 Structured version control for models? Version control the source of truth, which could be The model itself Stochastic model search + random seed Textual DSL A composition diagram with other models inserted Everything downstream of source of truth: deterministically cache Nix is current state-of-the-art. 2668#270unstable-270.xmlNotes202434Owen Lyncharia-0001false Originally, the idea was that we would version-control this data, directly. I still think that this is a good idea, but I think that there is a larger picture. Namely, you should version-control the source of truth for your scientific models. If that source of truth is a modeler directly editing the data of the model via some interface, then you should version-control the model. However, if the source of truth is an algorithm that did a model search to find that model, you should cache the model and attach it to a hash of this code+this random seed. If the source of truth is a handwritten textual DSL that gets compiled into the model, then you shouldn't check the result of that compilation process into version control. You should instead cache the model, keyed on the hash of the DSL. The state-of-the-art current system for caching the results of arbitrary computation is Nix. It is not the most ideal platform for a variety of reasons, including speed, but we have to at least learn from how it works if we want to do better. We can hack version control of structured data into Nix via hashing the patch application process. That is, we can make a Nix derivation with inputs the previous state and the patch which outputs the result of applying a patch to the previous state. Whether or not this is a good idea is not yet clear to me, but it is an idea. Interestingly enough, this would also be keyed on the hash of the versioning software, so every update to the versioning software would also cause all caches to be invalidated, and patches applied from scratch. Fortunately this is not too expensive: git does this every time you clone. Another sidenote: the difference between nix and content-addressed stores is that content-addressed stores are keyed by the hash of the thing that is stored, while nix is keyed by the hash of the inputs to a (more or less) pure function (the nix evaluator). 2676#273unstable-273.xmlInformal+Formal Collaboration: literate programming?Slide202434Owen Lyncharia-0001 I thought literate programming is dead... but is it? Rustdoc is literate programming 1lab is literate programming PBRT is literate programming Jupyter is literate programming Caching enables principled notebook computing Explainable AI involves AI... and explanations! 2678#272unstable-272.xmlNotes202434Owen Lyncharia-0001false Literate programming has a long history, starting with Knuth's implementation of TeX. One might argue that the traditional noweb style of literate programming has died out to some extent. But to a certain extent, if you consider notebooks or "documentation generators" like rustdoc to be literate programming, then literate programming is still going very strong. Also, the 1lab is a really interesting project: it combines formal mathematics with informal mathematics via literate agda. One of the Achilles heels of writing scientific documents as, say, Jupyter notebooks is that caching is hard. If you want a live-refreshing preview, then you can't rerun the notebook from scratch on every change. But if you want to avoid issues related to out-of-order cell execution, you have to rerun the notebook from scratch. And if you want to build notebooks in CI, then not having a cache system can mean >30 minute CI times, which is a massive drain on productivity. However, if your scientific system is built from the ground up to have fine-grained caching, including for simulation runs and figures, then it becomes practical to offer a live-reloading compilation server for your document, which only reruns cells when necessary. And by cache-sharing, someone else can edit your document without having to run your (potentially extremely expensive) simulations. You can also delegate computation to server farms and just pull in the result (link to post on this) Finally, a significant part of scientific modeling is informal explanation! Human language should be a first class citizen in the modeling workbench. 2680#274unstable-274.xmlConclusionSlide202434Owen Lyncharia-0001 Success requires scaling our development via more effective collaboration Informal collaboration needs to scale beyond couple of mathematicians write standalone paper Formal collaboration needs to scale beyond software package for single type of model in a single language Next steps: intertypes We shape technology for public benefit by advancing sciences of connection and integration. -- Topos Institute 3007lc-000Hlc-000H.xmlTips and TricksIf you haven't used forester before (which is most people), you can start by reading Forester for the Woodland Skeptic, which will get you started quickly and hopefully give you a sense of why to use forester.There are some useful macros for math/category theory in macros.tree. Hopefully as the forest grows, we will have some flame wars about notation and end up with some semi-productive standardization via this macros file.Comments (and by extension, automatic cross-posting to the forum) are opt-in. If you would like comments, then you must put at the top of your page.