Maybe Not - Rich Hickey

I bounce back and forth between TypeScript and Clojure. I've been keeping a close eye on Clojure's upcoming spec as it seems to be a nice opt-in, run-time type-checking solution. I like the flexibility and quickness with which dynamic languages let me iterate, but I want the safety. I want at least some of the benefits of both. So yes, they (dynamically typed languages) have some advantages. Sometimes when I'm iterating quickly, especially on new projects, I don't want to fight with the compiler. BUT, I vastly prefer TypeScript to JavaScript because JS has a lot of foot-guns. I think when we get the existential operator, it will have less foot-guns, but I will probably never again write a new project for work in JS from scratch. I really love the ideas Rich Hickey put forth in his most recent Clojure conj talk: [this video] The ability to spec out the shape of an object, and then in the different contexts in which you need some of that data, specify what you need, is an awesome solution to optionality (Watch the talk!). I absolutely love how much thought RH has put into the design of Clojure, and I think the final release of spec will be a close to perfect balance of dynamism and opt-in safety. Of course, TS is a superset of JS, so adopting it little by little sorta gets me that balance I want.

My mind blown experience was listening to a Rich Hickey talk (maybe not [1]) that flipped upside down my accepted method of thinking about types and type system paradigms. I still love writing Haskell and Rust, but I see the warts that he is so concisely pointing out. The either type really sticks out to me now, and I see the problems with overtyping what are essentially maps, and how rigid that makes every aspect of your programs. Currently I deal with a fairly large go monolith that had some clean code practices in it, and changing a single field that is going to make it into the public API ends up being hundreds or thousands of lines of changes. Such a waste of time to CRUD what is a map of strings. 1. [this video]

Since it's so close to this topic, I recommend Rich Hickey's talk from about a week ago, called "Maybe Not". It's mostly about the issue of how to specify the shape of data a function requires (takes as input) and provides (emits as output), but it also contains a really nice discussion of problems with Option/Maybe types. Anyway, my feeling is that I don't understand what people expect to happen here. Due to a misunderstanding, bad assumption, or other error, you expected to have something when you actually don't have the thing. A null-reference exception (or equivalent) is just the symptom. Null isn't the problem, our dumb brains that constantly make dumb mistakes are the problem. Option/Maybe does force you to deal with the case that you didn't get anything when you asked for something. But as Hickey points out in his talk, it isn't all benefit, there is a cost, and nobody ever talks about the cost. When I provide you an id and ask for a Book object in return, I want a Book back, not a `Maybe Book`. This concept does not map to anything in the real world, and it has problems in practical usage as well. (e.g. My function changes from maybe giving you a Book or maybe not, to definitely providing you a Book. So I change its return type from `Maybe Book` to `Book`. Except, oops, I just broke every single caller of my function, even though I strengthened the guarantee of the data my function will provide. That shouldn't break anyone.) [this video] (Discussion of Option/Maybe starts

Fair enough, I thought it was snark so I responded with snark. I think you should try both. Lisp as a language gives you a lot more freedom than Clojure, but Clojure has a more consistent and elegant language design. Clojure focuses on mostly-pure functional programming with immutable data structures, while Lisp lets you do pretty much anything anywhere, including hacking deep into the runtime itself (Clojure lets you do this to some extent too, but not as much as Lisp). Some videos that are worth viewing, which together I think provide a good sense of how the languages differ: - "Common Lisp for the Curious Clojurian" by Alan Diepert. A very lucid side-by-side comparison of Clojure and Common Lisp; assumes some but doesn't really require much knowledge of Clojure. [link] - "Maybe Not" by Rich Hickey. An unusual but thoughtful take on "types" in programming, which I think cuts to the heart of what Clojure is all about. [this video] - "Immutable Conversations - Common Lisp" interview with Alejandro Serrano and Michał "phoe" Herda. Demonstrates the deep programmability and flexibility of the Common Lisp runtime, including its unique "condition system". [link]

> Putting “here’s the fields I’m borrowing in the body” into the signature, in order to conform with the Golden Rule, looks very odd. I think this would be an amazing feature, but it would require something like typescripts partial types, where essentially every object/struct consists as a collection type of atomic fieldname:field-type types.[1] Rich hickey[2] has a great talk on how existing type systems break in scenarios where they shouldn't break, e.g. relaxing what you accept as arguments should be a forward compatible change, and similarly increasing the guarantees that you provide on returned values should also be forward compatible. Currently they are not, but partial types are one piece to that puzzle imho. 1: [link] 2: [this video]

Rich Hickey (creator of the Clojure language) has a good talk "Maybe Not" that touches on these issues, with a nice way of explaining it [this video] The capnproto link explains it concretely in terms of a message bus example, which is useful. But more abstractly you can think of the shape of data (aka schema, names and types) and field presence (optional/required) as separate things [link] First, being valid or invalid with respect to a static type system is a GLOBAL property of program -- writing a type checker will convince you of that. And big distributed systems don't have such global properties: [link] If they did, they'd be small :) Namely you could just reboot the whole thing at once. You can't reboot say the entire Internet at once, and this also holds for smaller systems, like the ones at say Google (and I'm sure Cloudflare, etc.). So the idea is that the shape/schema is a GLOBAL property -- you never want two messages called foo.RequestX or two fields called "num_bar" with different types, ever. But optional/required is LOCAL property. It depends on what version of a schema is deployed in a particular binary. Inherently, you need to be able to handle a mix of inconsistent versions running simultaneously. --- To be pedantic, I woulds say "making illegal states unrepresentable" DOES apply, but you can't do it in a STATIC type system. Your Maybe<T> type is not useful for data that crosses process boundaries. A distributed system isn't a state machine. 1. Lamport showed us

Consider reading/watching Rich Hickey's talk Effective Programs [0][1] and Maybe Not [2][3] In [0] in particular there're slides 22/23, here is part of the transcript but makes more sense with the slides on: > And you can call them problems, and I'm going to call them the problems of programming. And I've ordered them here [...] I've ordered them here in terms of severity. And severity manifests itself in a couple of ways. Most important, cost. What's the cost of getting this wrong? At the very top you have the domain complexity, about which you could do nothing. This is just the world. It's as complex as it is. > > But the very next level is the where we start programming, right? We look at the world and say, "I've got an idea about how this is and how it's supposed to be and how, you know, my program can be effective about addressing it". And the problem is, if you don't have a good idea about how the world is, or you can't map that well to a solution, everything downstream from that is going to fail. There's no surviving this misconception problem. And the cost of dealing with misconceptions is incredibly high. > >So this is 10x, a full order of magnitude reduction in (?) severity before we get to the set of problems I think are more in the domain of what programming languages can help with, right? And because you can read these they'll all going to come up in a second as I go through each one on some slide so I'm not going to read them all out right now. But importantly t

Maybe Not is an excellent video in some ways, but I just watched it a few years later and it reminded me that as much as I love almost every Rich video, I do wish he'd spend some time reading a little deeper on type systems and theory. For example his note about "Categorical descriptions" and then the explanation about maps/and spec at 25:00[1] seems to indicate he's unfamiliar with the difference between extensional and intensional type theory. The core difference between the two is extensional type systems decide equality on the observable behavior of the output. For example if two functions take the same input, and produce the same output extensional type systems/theories say they are the same. The problem with this is type equality is then undecidable. Extensional type systems also struggle because there are more than one way things might be equal! For example two string might be equal if you ignore case, but unequal if you don't. So if we want our type system to check if two functions are equal under case insensitivity an extensional type system will struggle with this.[2][3] Now in fairness to Rich, these concepts come more from the math community, especially the notion of types being equal under different paths, and in the case of Homotopy type theory are burred in unapproachable language and concepts... even by mathematicians standards. He also talks about select from spec, and it not preventing you from passing types broader types, and the necessity of specifying dee