Extending the Smalltalk Syntax 4

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 4: Hybrid Compilation

Have you ever heard of the idea the creators of Smalltalk had for allowing any class to choose its compiler? To provide support for this classes respond to the #compiler message before the actual compilation is attempted. Why then, this capability hasn’t been exploited yet? Are you willing to investigate it further? OK, bear with me.

In Story 3 of this series we discussed how to inline TaggedNodes in a Smalltalk method. We mentioned several applications of this capability and took JSON as a basic example. Today we can take a similar approach and try to see how to do our exploration with JSON in mind. From there it will become fairly clear how to proceed in other cases. So, let’s put ourselves this objective: compile the following method in our Smalltalk dialect:

jsonCoordinates
  <json>
  ^{
      "latitude": 48.858093,
      "longitude": 2.294694
    }

Where to start? Here is the roadmap:

  1. Discuss the introduction of pragmas for enabling foreign compilation.
  2. Introduce the HybridCompiler class.
  3. Generate the hybrid method when there are no arguments.
  4. Introduce the ParametricString class.
  5. Generate the hybrid method when there are arguments.

Task 1: Discussion


Let’s start by noticing how our example above is slightly different from what we did in Story 3. Here <json> is not a tag but a pragma (there is no closing tag). We are using this pragma to make it clear that we will be using a foreign compiler.

A similar example with JavaScript

canSimulateCase
    <js>
    $scope.canSimulateCase = function(c) {
        return !$scope.isProcessingCase(c)
    };

One difference with tagged nodes is that here the entire method body is written in a foreign language. Why this variation is interesting? Because it will allow us to pass Smalltalk arguments to foreign methods. Like this:

jsonCoordinates: lat longitue: long
  <json>
  ^{
      "latitude": #lat,
      "longitude": #long
    }

meaning that the foreign source code will be generated dynamically.

Note that I’ve used # to mark what follows as an argument. We don’t want to replace every occurrence of 'lat' and 'long' with the arguments; do we?, so we need to tell where we want the replacements to happen. The reason for using # as a marker is that it presents (almost) no collision with foreign tokens.

Task 2: Hybrid Compiler


If we get back to our examples above, we will see that these methods have two parts: (1) a Smalltalk header including the pragma and (2) the foreign code. This accounts for hybrid compilation. We need to, at least, parse the beginning of the method to read the pragma that tells which compiler to pick, and then pass it the body. For doing all of this we will need the following class

Object
    subclass: #HybridCompiler
    instanceVariableNames: 'source smalltalk foreing method'
    classVariableNames: ''
    poolDictionaries: ''

The smalltalk ivar is initialized to the Smalltalk compiler, and foreign to the parser (or compiler) associated to the method’s pragma. When the source is set, the smalltalk compiler is used to read the pragma ('json' in our example). At this point the Registry (see Story 3) will provide us with the foreign parser. If there is no pragma or there is one which is not in the Registry, the compilation is entirely on smalltalk. The method ivar will hold the compilation result.

Task 3: Hybrid method


Once an instance of HybridCompiler has been initialized, it is time to compile the method. For now we will assume that there are no arguments (unary case).

HybriCompiler >> compile
  | cm |
  foreign isNil ifTrue: [^method := smalltalk compileMethod: source].
  ast := foreign parse: self body.
  cm := smalltalk compileMethod: self template.
  method := ForeignMethod from: cm.
  method
    sourceCode: source;
    foreignCode: ast format;
    foreignParser: foreign

There are several things to explain here:

  • The #body method, with the help of the smalltalk parser, answers with the foreign body, i.e., the part of the source code that comes after the pragma.
  • The #template method answers with the source code of the Smalltalk method that will actually be executed when the hybrid method is invoked.
  • The ForeignMethod class is a subclass of CompiledMethod that adds support to certain messages required from hybrid methods.
  • The #format message sent to the ast is optional. I would recommend including it because it is nice to have your foreign code formatted as soon as you save your method.

In the unary case we are now, the #template method has the following source code

template
  ^self selector , '
  #code.
  #parser.
  ^#code'

where #selector answers, with the help of the smalltalk compiler, the method’s selector; the following two symbols are placeholders for two slots in the literal frame that we will change below. Note that the compiled method will answer with the contents of the first literal slot.

ForeignMethod >> foreignCode: aString
  self literalAt: 1 put: aString
ForeignMethod >> foreignParser: aParser
  self literalAt: 2 put: aParser

The second literal holds the foreign parser, which may be needed for further processing (e.g., painting the source code).

Task 4: Macro Expansion


We will now address the case where the method has arguments which are inserted in the foreign code using # as the marker prefix. Let’s refer to these prefixed identifiers as hashed tokens.

What we need is to dynamically replace all hashed tokens with the corresponding arguments. For doing this we will introduce a new class named ParametricString, which will implement this transformation as a service.

Basically this new object takes aString including hashed tokens and a sequence of all possible tokens (in our case, the sequence of method arguments). Using this information the object produces a sequence of strings and indexes. The strings are the fragments between hashed tokens, the indexes refer to the sequence of hashed tokens. For instance if the inputs are:

  • 'hello #world, this is a #test'.
  • #('test' 'dummy' 'word')

the object should generate the following sequence:

#('hello ' 3 ', this is a ' 1 '.')

with the somewhat clearer Squeak-braces syntax, this would be

{'hello'. 3. ', this is a '. 1. '.'}

Later on, when the object is required to expand the tokens using actual arguments it will replace the indexes with the corresponding values, concatenating them all. The message to do this will be

aParametricString expandWith: arg1 with: arg2 ...

Task 5: Hybrid method with arguments


Since we have already worked on the unary case in Task 3, we only need to redo the #template method for the case where there are arguments.

The first change to consider is that what before was simply the selector, it is now a keyword message with the formal arguments. This can be simply accomplished with the help of the smalltalk compiler, so I will not go in detail here. We will just assume that self selector will answer with the signature of the method (i.e., 'keyword1: arg1 keyword2: arg2'…).

Since the source code will be now a bit more complex, I will use the #streamContents: constructor.

template
  | arguments processor |
  arguments := self arguments.
  processor := ParametricString from: self body tokens: arguments.
  ^String streamContents: [:strm |
    strm
      nextPutAll: self selector;
      crtab;
      nextPutAll: '| ast |';
      crtab;
      nextPutAll: '#code.';
      crtab;
      nextPutAll: '#parser.';
      crtab;
      nextPutAll: 'ast := #parser parse: (#code expandWith: '.
    arguments
      do: [:arg | strm nextPutAll: arg]
      separatedBy: [strm nextPutAll: ' with: '].
    strm
      nextPutAll: ').';
      crtab;
      nextPutAll: '^ast format']

A key remark here is that in order to connect processor with: #code we need to make sure we plug the processor in the first literal slot, where #code acts as its placeholder. This is achieved by a modification to the #compile method we saw in Task 3. Instead of sending ast format as the argument of foreignCode: we need to send

ParametricString from: code tokens: self arguments

Final words


As always, the code presented here is just a sketch of the actual code, which might end up being a bit more complex (but not too much!). Also, there are lots of details in my description for which I haven’t included any code at all. The reason is two-fold. Firstly, my goal is to provide a roadmap rather loadable code. Secondly, I don’t want to capture all the fun just for myself.

Extending the Smalltalk Syntax 3

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 3: Tagged Nodes

What do you do when you have to include JSON in a Smalltalk method? Something like this?

jsonCoordinates
  ^'{
      "latitude": 48.858093,
      "longitude": 2.294694
    }'

In other words, do you represent JSON data with plain strings? Wouldn’t it be nice to improve this? What if the compiler knew that this String should conform to a specific syntax? What if the Smalltalk browser knew how to format JSON strings? Or even color them for the sake of readability?

Before jumping to JSON strings, let’s step back and think of other cases that might be similar. Have you ever represented HTML as a plain String? What about CSS or even JavaScript? I’m sure you have faced situations where you inlined foreign code as a plain String in a method, keeping that information in your head rather than in the method, where it should naturally belong? Want to change this? Ok. Let’s do it.

Where to start? Here is the roadmap:

  1. Consider the introduction of tags for inlining foreign scripts.
  2. Introduce a new subclass of LiteralNode named TaggedNode .
  3. Consider the introduction of foreign parsers such as a JsonParser.
  4. Introduce a new class of AST node named ForeignNode.
  5. Process the body of the foreign script, according to its semantics.

Task 1: Smalltalk tags?


Before making a decision for tags, let’s see which other options do we have. In order to inline foreign scripts, we must tell the Smalltalk parser how to delimit them. There are several delimiters already taken in Smalltalk:

  • White space
  • Single and double quotes
  • Parenthesis and brackets (both square and curly)

Can we think of any other? Backticks are tempting. The problem is that they would only work for a single semantics. Say we decide to delimit JSON using backticks; how would we delimit HTML or CSS or JavaScript or Assembly or C, should the future bring a need for any of them?

We want flexibility and that’s why tags are a good choice.
Using tags we will be able to inline foreign code like this:

jsonCoordinates
  ^<json>
    {
      "latitude": 48.858093,
      "longitude": 2.294694
    }
  </json>

And how do we make sure that tags do not confuse the Smalltalk parser? To answer this question think of all the places where $< is misplaced in regards to the Smalltalk syntax:

  • On the right of assignments
  • When a message argument is expected
  • On the right of the return symbol

In other words, none of the following sequence of characters conforms to the Smalltalk syntax:

  • temp := <
  • 3 + <
  • self msg: <
  • ^<

See? Every potential syntax error is an opportunity for extending the syntax!

Of course, angle brackets <...> are already legal in the Smalltalk syntax as pragma delimiters. But pragmas are illegal when placed in assignments, arguments and returns. To be valid, they must start a Smalltalk statement. And this is precisely why we will forbid tags at the beginning of statements and restrict them to assignments, arguments and returns.

Task 2: Add the class for tagged nodes


A tagged node is a way of delimiting foreign code and as such it is a new type of literal. So, add a subclass of LiteralNode named TaggedNode. This subclass will add the tag ivar that will link its instances to their specific meaning.

As we depticted above, instances of TaggedNode need to be instantiated by the parser in the following four cases:

  • When parsing an argument of a keyword message
  • When parsing the argument of a binary message
  • When parsing the expression of an assignment
  • When parsing the expression of a return node

This means that we need to modify essentially four methods so that they now check whether the next character is $<. If it is not, the original code regains control. Otherwise, the code branches to a new method that will scan the TaggedNode or fail (if there is no closing tag, etc.).

I’ve used the verb to scan because in order to form a TaggedNode, we will need to scan the input at the character level (usually the parser deals with tokens provided by the scanner).

When scanning the opening tag we will need to read the input until $> is reached (issuing and error if it isn’t). This will give us the value for the tag ivar of the TaggedNode. At this time we will also know that the closing tag should be '/', tag. So we can read the body of the foreign code until '</', tag, '>' is found (error if not).

At this point we are ready to

Task 3: Decide how to process the foreign script


Now we have access to the body of the TaggedNode. What do we do with it? Well, this depends on the semantics we want to give it. In the case of JSON, for instance, it would be enough to parse it using a JSON parser, and then format it using a JSON writer. We could also paint it with colors and emphases, so to make it look great in our environment.

In other cases, such as the one where the foreign code is Assembly, we could decide to go a step further and compile it into machine code. This will bring two capabilities: (1) Parsing and formatting/painting the Assembly source code and (2) Making the node answer with the corresponding machine code when the method is executed.

There are many other possibilities. In the case of JavaScript or any other programming language, we could decide to execute it on top of Smalltalk (at least up to some extent, this should be feasible).

On the other end of our wide horizon of possibilities there is one that consists in doing nothing, i.e, simply keeping the body as a String with no special semantics. This is useful for experimentation. For instance, if you plan to write a parser for inlining VBA code, before embarking in such a project, you might want to see how
the tagged code would look. You will need to format it yourself and will have no coloring available. However, it will bring a secondary benefit: you will not have to worry about duplicating embedded quotes (in VBA the quote is the comment separator).

The last case is the simpler one. Still, it requires the introduction of a new kind of literal node, which we will call StringNode. So, instead of keeping the body in the TaggedNode as a String, we will create a StringNode with the body as its value and will keep this new node as the value of the TaggedNode. This indirection will provide the flexibility we need for making free use of TaggedNode.

Note also that while making these decisions you should keep in mind that sometimes it is not necessary to implement a parser of the entire specification of the foreign language. For instance, if you will only deal with CTypes and C-Structures, you don’t need to parse arbitrary C, you just need to parse these declarations. The same might be true with other languages. Usually you will only inline a limited subset of them. The key here is to create the machinery that will make further enhancements easier and consistent.

Task 4: The foreign node


So far we have discussed two new nodes: TaggedNode and StringNode, both subclasses of LiteralNode. The ivar tag in TaggedNode holds the node’s tag string. Where we have to be careful is in deciding the contents of the value ivar because this is where the semantics enters the game.

Since we are planning for support of different languages, we will need a global Registry of available parsers/compilers. For instance, the package that loads the JSON parser will be able to register the <json> tag with the corresponding parser. Similarly for <html>, <css>, <asm>, <js>, <vba>, etc.

In this way, when the TaggedNode receives the #body: message with the foreign code as the argument, it will be able to enter the Registry with its tag and get the corresponding parser from there. If there is none, the TaggedNode will resort to StringNode, passing it the body and keeping this node in its value ivar.

TaggedNode >> body: aString
  value := Registry
    at: tag
      ifPresent: [:p | ForeignNode new parser: p]
      ifAbsent: [StringNode new].
  value value: aString

The ForeignNode will have two ivars: parser and ast. The latter is computed as follows:

ForeignNode >> value: aString
  ast := parser parse: aString

Task 5: Compile/Process


Now that we have all the pieces in place we can use them to, at least, format and/or color the foreign code. This is simply achieved by asking the ForeignNode, its ast or its formatted/colored representation. You could also provide more advanced featrues such as the ones we have mentioned in Task 3, or even take advantage of yet another technique that we will discuss in the next story which is Hybrid Compilation.

Extending the Smalltalk Syntax 2

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 2: Supporting Pragmas

All Smalltalk dialects support pragmas. Here is one:

identityHash
  <primitive: Hash>
  ^0

However, not all of them support other types of pragmas, as Pharo does. So, let’s see what it would take to add support for them to our dialect.

Where to start? Here is the roadmap:

  1. Find the Smalltalk Parser in your system
  2. In the Smalltalk Parser find the place where primitives are parsed
  3. Modify the code that parses primitive declarations to recognize pragmas as well
  4. Find a place in the CompiledMethod to store the pragma annotation

Now let’s see some hints on how to accomplish the tasks above:

Task 1: Find the Smalltalk Parser


Look for selectors like parse, parse:, parseMethod:, parseLiteral:, etc. In some dialects the entry point to parsing services is the Compiler, so you can debug the initial steps required to compile any expression until you reach the parser. You can also check to see if there are implementors of parserClass.

Task 2: Find the parsing of primitive declarations


This is best accomplished by debugging the compilation of any method with a primitive declaration. Just create a primitive method and debug its compilation. After some few clicks you should get to the parser method that distinguishes it as primitive.

Task 3: Parse the pragma


For this task I would recommend starting simple. There will be time for further sophistication. In this case “simple” means: assume the pragma is just one token. Much like:

methodWithPragma
  <ourPragma>
  self doSomething; etc.

The existing code in the parser should complain if the token that comes after $< is not 'primitive:' (or friends). And this is precisely where we need to branch, save the token as our pragma and (in this first attempt) save it in the MethodNode. Note that there is no need to add a new ivar to MethodNode, we can save the token in the same ivar where primitives are held. After all, primitives are numbers while pragmas are not, so we will be able to distinguish between them.

Don’t forget to consume the closing $>, which should be the next token. Otherwise issue a parsing error.

Task 4: Save the pragma in the method


This step is actually not required as we could always resort to the AST for getting the pragma. One could argue that it would be too slow to parse the entire method every time we want to access its pragma, should it have any. However, since pragmas occur before any other statements, there is no need to build the entire AST. In fact, we would only need a service in the parser that will stop parsing right before the parsing of the method sentences starts.

However, if you insist on avoiding any parsing, when a new CompiledMethod is compiled, the MethodNode should somehow inject the pragma in it. One way to do this is to add the pragma as the first literal. A practice also known as stealing literals. However, if we steal the first literal, how are we going to tell whether this first literal is a pragma or not?

There might be several tricks for solving this. For instance, in most dialects 0 (the SmallInteger) is never saved in the literal frame. The reason is that there are special bytecodes to operate with it, so the constant 0 doesn’t need to go into the literal frame.

Therefore, we could add the pragma as the first literal, and then add 0 as the second. Thus, in order to check and retrieve the pragma of a method we would do the following:

CompiledMethod >> pragma
  literals := self literals.
  ^(literals size >= 2 and: [(literals at: 2) = 0])
    ifTrue: [literals at: 1]

Note that the method will answer with nil if it has no pragma.

Final words


To make sure that 0 doesn’t go to the literal frame in your dialect, evaluate the following:

CompiledMethod allInstances select: [:cm | cm literals includes: 0]

It should answer with an empty array. If not, you will need to find another trick (hint: add two 0s instead of one.)

The simple case I’ve depicted here is for adding support to unary pragmas. You might want to allow for binary and keyword pragmas as well. The idea is the same. Just keep reading more tokens in Task 3, until $> is reached. Take a look at how Pharo does this for inspiration, starting at RBParser >> parsePragma. Then adapt the idea to your case.

If you decide to add support for multiple pragmas, note that you will need to steal n + 1 literals rather than 2 as we did in Task 4. You only need to move the 0 right after the last injection in the literal frame.

Extending the Smalltalk Syntax

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 1: Adding Squeak Braces

Does your dialect support Squeak Braces? To answer this question, try to evaluate the following expression:

{Date today. 3 + 4}

If you get an Array with the above two elements, today’s date and 7, it does. Otherwise, you would get a compilation error. In the latter case, you might be interested in extending the syntax of your dialect so to have what I call Squeak Braces.

Where to start? Here is the roadmap:

  1. Find the hierarchy of classes that model parse tree nodes.
  2. Add a new class for modeling brace arrays (let’s call it BraceNode or ArrayNode).
  3. Add an ivar to the class for holding the elements of the array (let’s call it elements).
  4. Add a method to the class for transforming the elements into a cascade node (see below).
  5. Add all other required methods that nodes need to implement, especially those that message nodes have.

Now let’s see some hints on how to accomplish the tasks above:

Task 1: Finding the AST hierarchy


The Abstract Syntax Tree (a.k.a. AST) is the object that models the decomposition of a piece of source code into its constituents. This typically includes nodes for both the whole method and also its parts: literals, blocks, variables, etc. Therefore I would start by trying to find a class that includes Literal or LiteralNode in its name. From this class, go to the top of the hierarchy and you will get the complete picture of the place where you will be working next.

Task 2: Adding the new class


It’s simple, just subclass from the root of the AST hierarchy a new class appropriately named, say, BraceNode.

Task 3: Adding the required ivars


For sure we will need an ivar to keep the elements of the array. So, add the elements ivar. We will likely add one more ivar later though.

Task 4: Transforming the node into a cascade node


The idea here is to add a new method, say asCascadeNode, whose job is to answer with the CascadeNode that results from the expression:

(Array with: n)
  at: 1 put: (elements at: 1);
  …;
  at: n put: (elements at: 2);
  yourself

where n is elements size. To do this, you need to find the CascadeNode in the AST hierarchy and become familiar with it so you can create one instance of it for the method to return.

Task 5: Add other required methods


Typically, AST nodes implement the #acceptVisitor: message for supporting the Visitor pattern’s double dispatching mechanism. It’s implemenation is straightforward:

acceptVisitor: aVisitor
  aVisitor acceptBraceNode: self

You also need to find all visitors that you will need to enhance. To discover them look for implementors of visiting messages in the AST hierarchy such as visitCascadeNode:, etc. Writing each of the required implementor of visitCascadeNode: is also straightforward:

visitBraceNode: aBraceNode
  self visitCascadeNode: aBraceNode asCascadeNode

For other messages, take inspiration from the other nodes, especially, from CascadeNode.

Improvements


Once you have all of this complete and tested, you may want to improve your implementation a little bit. For instance, some occurrences of these Squeak Braces are just literals. One example would be:

{3. $a. {'hello' 'world'}}

This is actually equivalent to:

#(3. $a. #('hello' 'world'))

However, our implementation would work as well if we had written:

(Array new: 3)
  at: 1 put: 3;
  at: 2 put: $a;
  at: 3 put: (
    (Array new: 2)
      at: 1 put: 'hello';
      at: 2 put: 'world';
      yourself);
  yourself

which sends 9 messages instead of none! To avoid this waste, what we can do is to give literal arrays a special treatment. Here is how:

Task 6


At the top of the AST hierarchy, add the method isLiteral returning false (this might or might not be there already). Now repeat the same for LiteralNode except that this time answer with true.
Finally, add the isLiteral method to BraceNode on the lines of:

isLiteral
   ^elements conform: [:e | e isLiteral]

given that the elements of our BraceNode are themselves instances of AST nodes, this closes the circle.

Task 7


Next, add the asLiteralNode method to BraceNode on the lines of:

asLiteralNode
  ^LiteralNode new
    value: self literal;
    start: self start;
    stop: self stop

So, the only piece that is missing is the #literal message. Here it is:

literal
  ^elements collect: [:e | e literal]

where

LiteralNode >> #literal
  ^self value

Task 8


Finally, modify your implementations of visitBraceNode: method like this:

visitBraceNode: aBraceNode
  aBraceNode isLiteral
    ifTrue: [self visitLiteralNode: aBraceNode asLiteralNode]
    ifFalse: [self visitCascadenode: aBraceNode asCascadNode]

Depending on the internals of your system, some additional tweaks might be required. Just make sure your testing coverage is high enough. I almost forgot! In Task 3, I said that we would likely add another ivar to BraceNode, remember? Well, I was thinking in a cache for the result of asCascadeNode or asLiteralNode, depending on the case. This will help to preserve the identity of the BraceNode substitute should the translation be required more than once. No need to do this in advance though. Just keep it in mind.

A Look Ahead to Programming Languages in 2020

Python and JavaScript are the two hottest programming languages today. However, they cannot remain on top forever. Eventually, they must fall out of favour, as all languages do. This is likely to happen within the next decade or so. What languages might come to replace them? Here’s my list of challengers…

Dart

Thanks to the Flutter framework and Google’s imprimatur, this language has quickly risen in popularity. It’s similar to the same driving force that made Ruby so popular: the Rails framework. And if Google’s Fuchsia takes off, Dart will be in the centre of it.

Key advantage: it’s a much better language than JavaScript.

Key disadvantage: it’s up against JavaScript and its deluded hordes.

Mandelbrot set sample:

class Complex {
  double _r,_i;
 
  Complex(this._r,this._i);
  double get r => _r;
  double get i => _i;
  String toString() => "($r,$i)";
 
  Complex operator +(Complex other) => new Complex(r+other.r,i+other.i);
  Complex operator *(Complex other) =>
      new Complex(r*other.r-i*other.i,r*other.i+other.r*i);
  double abs() => r*r+i*i;
}
 
void main() {
  double start_x=-1.5;
  double start_y=-1.0;
  double step_x=0.03;
  double step_y=0.1;
 
  for(int y=0;y<20;y++) {
    String line="";
    for(int x=0;x<70;x++) {
      Complex c=new Complex(start_x+step_x*x,start_y+step_y*y);
      Complex z=new Complex(0.0, 0.0);
      for(int i=0;i<100;i++) {
        z=z*(z)+c;
        if(z.abs()>2) {
          break;
        }
      }
      line+=z.abs()>2 ? " " : "*";
    }
    print(line);
  }
}

Elixir

Elixir is an Erlang-derivative with an improved syntax and the same, amazing support for concurrency. As a pure functional language, it has a good likelihood of elevating this paradigm into the mainstream.

Key advantage: it makes functional programming exceptionally easy. And it’s great for concurrency.

Key disadvantage: you need to understand the underlying OTP foundation which can be a daunting task.

Mandelbrot set sample:

defmodule Mandelbrot do
  def set do
    xsize = 59
    ysize = 21
    minIm = -1.0
    maxIm = 1.0
    minRe = -2.0
    maxRe = 1.0
    stepX = (maxRe - minRe) / xsize
    stepY = (maxIm - minIm) / ysize
    Enum.each(0..ysize, fn y ->
      im = minIm + stepY * y
      Enum.map(0..xsize, fn x ->
        re = minRe + stepX * x
        62 - loop(0, re, im, re, im, re*re+im*im)
      end) |> IO.puts
    end)
  end
 
  defp loop(n, _, _, _, _, _) when n>=30, do: n
  defp loop(n, _, _, _, _, v) when v>4.0, do: n-1
  defp loop(n, re, im, zr, zi, _) do
    a = zr * zr
    b = zi * zi
    loop(n+1, re, im, a-b+re, 2*zr*zi+im, a+b)
  end
end
 
Mandelbrot.set

Golang

Another Google-supported language, Golang has proven to be winner, thanks to its lightning-quick compilation speed, easy and efficient concurrency, and remarkable simplicity. The only thing missing is generics, and this feature is on the roadmap.

Key advantage: it’s super-simple, and great for concurrency.

Key disadvantage: it lacks generics (for now).

Mandelbrot set sample:

package main
 
import (
    "fmt"
    "image"
    "image/color"
    "image/draw"
    "image/png"
    "math/cmplx"
    "os"
)
 
const (
    maxEsc = 100
    rMin   = -2.
    rMax   = .5
    iMin   = -1.
    iMax   = 1.
    width  = 750
    red    = 230
    green  = 235
    blue   = 255
)
 
func mandelbrot(a complex128) float64 {
    i := 0
    for z := a; cmplx.Abs(z) < 2 && i < maxEsc; i++ {
        z = z*z + a
    }
    return float64(maxEsc-i) / maxEsc
}
 
func main() {
    scale := width / (rMax - rMin)
    height := int(scale * (iMax - iMin))
    bounds := image.Rect(0, 0, width, height)
    b := image.NewNRGBA(bounds)
    draw.Draw(b, bounds, image.NewUniform(color.Black), image.ZP, draw.Src)
    for x := 0; x < width; x++ {
        for y := 0; y < height; y++ {
            fEsc := mandelbrot(complex(
                float64(x)/scale+rMin,
                float64(y)/scale+iMin))
            b.Set(x, y, color.NRGBA{uint8(red * fEsc),
                uint8(green * fEsc), uint8(blue * fEsc), 255})
 
        }
    }
    f, err := os.Create("mandelbrot.png")
    if err != nil {
        fmt.Println(err)
        return
    }
    if err = png.Encode(f, b); err != nil {
        fmt.Println(err)
    }
    if err = f.Close(); err != nil {
        fmt.Println(err)
    }
}

Julia

Julia’s strength is its excellent support for mathematical computation. The math-friendly syntax is great for data scientists. If any language can overthrow Python, this one is definitely a contender.

Key advantage: it’s well-designed for scientists.

Key disadvantage: it’s up against Python, the king of data science.

Mandelbrot set sample:

using Images
 
@inline function hsv2rgb(h, s, v)
    const c = v * s
    const x = c * (1 - abs(((h/60) % 2) - 1))
    const m = v - c
 
    const r,g,b =
        if h < 60
            (c, x, 0)
        elseif h < 120
            (x, c, 0)
        elseif h < 180
            (0, c, x)
        elseif h < 240
            (0, x, c)
        elseif h < 300
            (x, 0, c)
        else
            (c, 0, x)
        end
 
    (r + m), (b + m), (g + m)
end
 
function mandelbrot()
 
    const w, h = 1000, 1000
 
    const zoom  = 0.5
    const moveX = 0
    const moveY = 0
 
    const img = Array{RGB{Float64}}(h, w)
    const maxIter = 30
 
    for x in 1:w
        for y in 1:h
            i = maxIter
            const c = Complex(
                (2*x - w) / (w * zoom) + moveX,
                (2*y - h) / (h * zoom) + moveY
            )
            z = c
            while abs(z) < 2 && (i -= 1) > 0
                z = z^2 + c
            end
            const r,g,b = hsv2rgb(i / maxIter * 360, 1, i / maxIter)
            img[y,x] = RGB{Float64}(r, g, b)
        end
    end
 
    save("mandelbrot_set.png", img)
end
 
mandelbrot()

Kotlin

Kotlin is the better Java. In fact, it’s practically a drop-in replacement for Java. Google have already made it a first-class language for Android development.

Key advantage: it’s a souped-up Java.

Key disadvantage: it’s a very large language, even compared to Java.

Mandelbrot set sample:

import java.awt.Graphics
import java.awt.image.BufferedImage
import javax.swing.JFrame
 
class Mandelbrot: JFrame("Mandelbrot Set") {
    companion object {
        private const val MAX_ITER = 570
        private const val ZOOM = 150.0
    }
 
    private val img: BufferedImage
 
    init {
        setBounds(100, 100, 800, 600)
        isResizable = false
        defaultCloseOperation = EXIT_ON_CLOSE
        img = BufferedImage(width, height, BufferedImage.TYPE_INT_RGB)
        for (y in 0 until height) {
            for (x in 0 until width) {
                var zx = 0.0
                var zy = 0.0
                val cX = (x - 400) / ZOOM
                val cY = (y - 300) / ZOOM
                var iter = MAX_ITER
                while (zx * zx + zy * zy < 4.0 && iter > 0) {
                    val tmp = zx * zx - zy * zy + cX
                    zy = 2.0 * zx * zy + cY
                    zx = tmp
                    iter--
                }
                img.setRGB(x, y, iter or (iter shl 7))
            }
        }
    }
 
    override fun paint(g: Graphics) {
        g.drawImage(img, 0, 0, this)
    }
}
 
fun main(args: Array<String>) {
    Mandelbrot().isVisible = true
}

Lua

Key advantage: Lua is a small, simple, fast, embeddable, portable, and flexible language.

Key disadvantage: it has been overlooked for 26 years. What’s going to change now?

Mandelbrot set sample:

local maxIterations = 250
local minX, maxX, minY, maxY = -2.5, 2.5, -2.5, 2.5
local miX, mxX, miY, mxY
function remap( x, t1, t2, s1, s2 )
    local f = ( x - t1 ) / ( t2 - t1 )
    local g = f * ( s2 - s1 ) + s1
    return g;
end
function drawMandelbrot()
    local pts, a, as, za, b, bs, zb, cnt, clr = {}
    for j = 0, hei - 1 do
        for i = 0, wid - 1 do
            a = remap( i, 0, wid, minX, maxX )
            b = remap( j, 0, hei, minY, maxY )
            cnt = 0; za = a; zb = b
            while( cnt < maxIterations ) do
                as = a * a - b * b; bs = 2 * a * b
                a = za + as; b = zb + bs
                if math.abs( a ) + math.abs( b ) > 16 then break end
                cnt = cnt + 1
            end
            if cnt == maxIterations then clr = 0
            else clr = remap( cnt, 0, maxIterations, 0, 255 )
            end
            pts[1] = { i, j, clr, clr, 0, 255 }
            love.graphics.points( pts )
        end
    end
end
function startFractal()
    love.graphics.setCanvas( canvas ); love.graphics.clear()
    love.graphics.setColor( 255, 255, 255 )
    drawMandelbrot(); love.graphics.setCanvas()
end
function love.load()
    wid, hei = love.graphics.getWidth(), love.graphics.getHeight()
    canvas = love.graphics.newCanvas( wid, hei )
    startFractal()
end
function love.mousepressed( x, y, button, istouch )
    if button ==  1 then
        startDrag = true; miX = x; miY = y
    else
        minX = -2.5; maxX = 2.5; minY = minX; maxY = maxX
        startFractal()
        startDrag = false
    end
end
function love.mousereleased( x, y, button, istouch )
    if startDrag then
        local l
        if x > miX then mxX = x
        else l = x; mxX = miX; miX = l
        end
        if y > miY then mxY = y
        else l = y; mxY = miY; miY = l
        end
        miX = remap( miX, 0, wid, minX, maxX ) 
        mxX = remap( mxX, 0, wid, minX, maxX )
        miY = remap( miY, 0, hei, minY, maxY ) 
        mxY = remap( mxY, 0, hei, minY, maxY )
        minX = miX; maxX = mxX; minY = miY; maxY = mxY
        startFractal()
    end
end
function love.draw()
    love.graphics.draw( canvas )
end

Pharo

Pharo is a modern variant of Smalltalk, a remarkably productive object-oriented language. In fact, Smalltalk is the paragon of OOP and has inspired nearly every other OOP language on the planet. In the end, no language does OOP better than Smalltalk.

Pharo is also one of the simplest, most elegant languages in the world. You can actually learn the entire syntax of Smalltalk within 15 minutes!

Key advantage: it’s super-productive, like 5X more productive!

Key disadvantage: it requires a different programming mindset. People are afraid of change.

Fractal tree sample (based on Squeak):

Object subclass: #FractalTree
    instanceVariableNames: ''
    classVariableNames: ''
    poolDictionaries: ''
    category: 'RosettaCode'

"Methods for FractalTree class"

tree: aPoint length: aLength angle: anAngle
    | p a |
 
    (aLength > 10) ifTrue: [
        p := Pen new.
        p up.
        p goto: aPoint.
        p turn: anAngle.
        p down.
        5 timesRepeat: [
            p go: aLength / 5.
            p turn: 5.
        ].
        a := anAngle - 30.
        3 timesRepeat: [
            self tree: p location length: aLength * 0.7 angle: a.
            a := a + 30.
        ]
    ].
 
draw
    Display restoreAfter: [
        Display fillWhite.      
        self tree: 700@700 length: 200 angle: 0.
    ]

"Execute"

FractalTree new draw.

Rust

Rust has gained recognition for its memory safety feature: the borrow checker. This feature practically eliminates the entire class of memory-related programming errors. Rust promises much safer programming.

Key advantage: it helps make software much more reliable.

Key disadvantage: it’s tough to learn, and the borrow checker can be complicated to understand.

Mandelbrot set sample:

extern crate image;
extern crate num_complex;
 
use std::fs::File;
use num_complex::Complex;
 
fn main() {
    let max_iterations = 256u16;
    let img_side = 800u32;
    let cxmin = -2f32;
    let cxmax = 1f32;
    let cymin = -1.5f32;
    let cymax = 1.5f32;
    let scalex = (cxmax - cxmin) / img_side as f32;
    let scaley = (cymax - cymin) / img_side as f32;
 
    // Create a new ImgBuf
    let mut imgbuf = image::ImageBuffer::new(img_side, img_side);
 
    // Calculate for each pixel
    for (x, y, pixel) in imgbuf.enumerate_pixels_mut() {
        let cx = cxmin + x as f32 * scalex;
        let cy = cymin + y as f32 * scaley;
 
        let c = Complex::new(cx, cy);
        let mut z = Complex::new(0f32, 0f32);
 
        let mut i = 0;
        for t in 0..max_iterations {
            if z.norm() > 2.0 {
                break;
            }
            z = z * z + c;
            i = t;
        }
 
        *pixel = image::Luma([i as u8]);
    }
 
    // Save image
    let fout = &mut File::create("fractal.png").unwrap();
    image::ImageLuma8(imgbuf).save(fout, image::PNG).unwrap();
}

TypeScript

TypeScript is JavaScript…with benefits. It primarily adds static typing. Compatibility with JavaScript makes it a favourite of front-end web developers because they already know JavaScript and they hardly need to alter their workflow.

Key advantage: it’s JavaScript, so there’s no big change for JavaScript developers.

Key disadvantage: it’s still JavaScript, so you inherit all of its baggage.

Fractal tree sample:

// Set up canvas for drawing
var canvas: HTMLCanvasElement = document.createElement('canvas')
canvas.width = 600
canvas.height = 500
document.body.appendChild(canvas)
var ctx: CanvasRenderingContext2D = canvas.getContext('2d')
ctx.fillStyle = '#000'
ctx.lineWidth = 1
 
// constants
const degToRad: number = Math.PI / 180.0
const totalDepth: number = 9
 
/** Helper function that draws a line on the canvas */
function drawLine(x1: number, y1: number, x2: number, y2: number): void {
    ctx.moveTo(x1, y1)
    ctx.lineTo(x2, y2)
}
 
/** Draws a branch at the given point and angle and then calls itself twice */
function drawTree(x1: number, y1: number, angle: number, depth: number): void {
    if (depth !== 0) {
        let x2: number = x1 + (Math.cos(angle * degToRad) * depth * 10.0)
        let y2: number = y1 + (Math.sin(angle * degToRad) * depth * 10.0)
        drawLine(x1, y1, x2, y2)
        drawTree(x2, y2, angle - 20, depth - 1)
        drawTree(x2, y2, angle + 20, depth - 1)
    }
}
 
// actual drawing of tree
ctx.beginPath()
drawTree(300, 500, -90, totalDepth)
ctx.closePath()
ctx.stroke()

WebAssembly

WebAssembly is a dark horse. In the next decade or so, it could possibly spawn a number of languages that rise to the top. WebAssembly is only a compilation target, but there’s no reason it couldn’t spread far beyond the web domain. What WebAssembly-based language(s) could rise to the top? It’s anybody’s guess.

(Previously published at Hacker Noon.)

How is Smalltalk’s revival like the e-car’s revival?

It’s not a riddle. I’ll give you the answer.

There are several interesting parallels between the evolution of the electric car and the evolution of Smalltalk. Let’s look at e-cars first.

E-cars were pioneered in the 1830s. By 1900, they had reached their peak of popularity with a third of all cars manufactured in the United States being electric.

By 1920, the maturity and popularity of the internal combustion engine had made e-cars no longer viable. So e-cars faded away.

By the 1960s, however, rising gas prices and concerns over pollution caused people to re-examine the electric car. But it would be another three decades before the Toyota Prius made electric viable again, if only in the form of hybrids.

The big breakthrough for true e-cars came in 2006 with Elon Musk’s Tesla Roadster. Everybody wanted one.

And today, Tesla remains the e-car that everybody wants.

Now, let’s look at Smalltalk. Smalltalk was pioneered in the 1970s. (The first official language release was Smalltalk-80 in 1980.)

Smalltalk had reached its peak of popularity in the early 1990s when it became the second most popular object-oriented language in the world (C++ was #1). Even IBM chose it as the centrepiece of their VisualAge enterprise initiative.

By 1995, that began to change. Sun Microsystems’ Java language burst onto the scene and sucked all the oxygen out of the room. Smalltalk was among the casualties.

By 2008, however, a modern variant of Smalltalk known as Pharo was birthed. It arrived at the right time because the world was suffering from low programmer productivity and a backlog of software applications begging to be written. The world was also suffering from a plethora of overly complex programming languages like C++, Java, C#, D, Groovy, and Scala. This group would be later joined by JavaScript, Kotlin, Rust, Swift, and TypeScript.

People are clamouring for fresh air. They want simpler languages. That’s one reason for the resurgence of Python (though calling Python a “simple” language is rather dubious).

It’s also why Golang took off, and why Dart, Elixir, Lua, and Nim are beloved.

Today, Pharo is a very exciting language. It has been innovating by leaps and bounds over the past decade. One of the most notable developments is the Glamorous Toolkit.

But Smalltalk is a family of languages, so we should also give a shout out to other great Smalltalk flavours such as GemStone/S from GemTalk Systems, VA Smalltalk from Instantiations, and VisualWorks from Cincom. Not to be overlooked are Amber (transpiles to JS), Cuis Smalltalk, Dolphin Smalltalk (optimized for Windows), GNU Smalltalk (for command line lovers), Hoot Smalltalk (JVM support), and Squeak.

Pharo is also extremely versatile which is well-explained in this article, “Smalltalk: It’s Not Your Grandparent’s Programming Language.”

Let me close by pointing out another interesting parallel. In the early years of the e-car, there were technical obstacles that limited its popularity. E-cars had a very limited range. They couldn’t compete with low-cost gas. They had limited horsepower.

In the early years of Smalltalk, it required expensive hardware to run on (processor and memory). It needed high-resolution graphical displays. It couldn’t compete with C/C++ and FORTRAN and BASIC and Pascal.

Today, both are experiencing a revival. E-cars are more economical than ever. Smalltalk can run on devices as small as the Raspberry Pi, and run well!

The epitome of the e-car is Tesla. It is the most aspirational electric car in the world.

The epitome of programming languages for productivity and ease of use is Pharo/Smalltalk. It, too, is aspirational. Is there a programmer alive who doesn’t want to write their applications quickly and easily, like in half the time? I don’t know about you, but I don’t want to work harder than I have to.

(Previously published at gitconnected.)

Smalltalk: It’s Not Your Grandparent’s Programming Language

Smalltalk is old. Very old. Just like LISP is very old. Just like Forth is very old. Just like Erlang is very old. Just like Haskell is very old.

Let’s face it, any language created before 1995 is probably one that your parent, or even grandparent, used in their IT career. (I’m speaking to the current generation.)

Old programming languages are considered not cool. They’re not dope. Nobody wants to be like their grandparent.

This is a very unfortunate sentiment. It is couched in ignorance. It is myopic and parochial. The sentiment blinds you to wonderful opportunities.

You see, old languages are routinely updated for modern times. LISP can be found in Clojure (2007). Forth can be found in Factor (2003). Erlang can be found in Elixir (2011). And although Haskell is old, it is considered new, or at least a rediscovered treasure.

And so it is with Smalltalk. The language is timeless. It is as fresh today as it was 40 years ago (with the release of Smalltalk-80).

Smalltalk promises a productivity gain that is almost revolutionary, thanks to its unique and synergistic combination of simple, concise language, easily accessible live coding IDE, and a persistable system of live objects. Smalltalkers regularly claim a productivity gain of 5X.

Smalltalk is remarkably versatile. You can do back-end web with Seaside and Teapot frameworks. You can do front-end web with Amber and PharoJS transpiled languages. You can do mobile with Cordova/PhoneGap. You can do data science with PolyMath library, Roassal data visualization, and Moose data analytics platform. You can do machine learning with TensorFlow and Keras. You can do IoT with PharoThings. You can do robotics with PhaROS. You can do virtual reality. You can even script the Unreal game engine!

Roassal demonstration
virtual reality demonstration
Unreal game engine demonstration

This is all cutting-edge stuff. All of it was unimaginable 40 years ago. So let’s drop the grandparent attitude, shall we?

Maserati is your grandparent’s sports car. But who wouldn’t want to drive a modern Maserati?!

(Previously published at gitconnected.)

Why Smalltalk is so easy to evangelize

Of all the programming languages in the world, Smalltalk is, by far, the easiest one to evangelize. Why is that?

First, Smalltalk has a wonderful historical legacy. It has contributed an enormous amount to the software industry, to computer science, and to programming language design. It pioneered the language virtual machine and JIT compilation. It created the world’s first modern IDE. It made live coding easily accessible. It introduced the MVC architectural pattern. It was largely responsible for TDD (test-driven development). It pioneered the development of object databases. It gave us the first refactoring browser. It was instrumental in developing GUI and WYSIWYG.

It influenced the design of Objective-C, Ruby, PHP, Perl, Python, Groovy, Scala, Dart, and other languages.

It popularized object-oriented programming (whereas Simula 67 academically introduced the concept of classes and objects). Remember the famous August 1981 cover of BYTE magazine?

Second, Alan Kay and his group at Xerox PARC are legendary. Alan Kay’s vision was extraordinary. His object-oriented philosophy was brilliant. It was far superior to the philosophy espoused by people like Bjarne Stroustrup.

Alan Kay on the early days of Smalltalk (jump to 2:18)

Third, Smalltalk has a wonderful track record. It actually achieved a significant measure of popularity in the 1980s and 1990s. It was adopted by IBM for their VisualAge enterprise initiative. It was used by many prominent customers, including the U.S. joint military, JP Morgan, Orient Overseas Container Lines, Desjardins, UBS, Texas Instruments, Telecom Argentina, BMW, and Siemens AG.

It has a very significant commercial vendor presence which enterprise companies appreciate. Look at Cincom (VisualWorks), Instantiations (VA Smalltalk), and GemTalk Systems (GemStone/S). It has significant corporate sponsorship from the Pharo Consortium.

Fourth, Smalltalk has statistical evidence (in the Namcook Analytics study by Capers Jones) to support the claim of enormous programmer productivity. This is consistent with much anecdotal evidence for the same claim.

Fifth, Smalltalk has a rich family of languages like Pharo, Squeak, Cuis Smalltalk, Dolphin Smalltalk (optimized for Windows), GNU Smalltalk (for command line lovers), Amber (transpiles to JS), Hoot Smalltalk (JVM support), VisualWorks, VA Smalltalk, and GemStone/S (high-performance object databases) that can satisfy a myriad of needs.

Moreover, Pharo is the fastest evolving and innovating Smalltalk in history! It has remade itself in just under a decade.

Sixth, Smalltalk makes object-oriented programming very easy, much easier than in C++, Java, C#, and others. Smalltalk is supremely simple and easy to learn. Its complete syntax fits on a post card. You can learn all of Smalltalk’s syntax within 15 minutes!

Seventh, Smalltalk is remarkably versatile. You can do back-end web with Seaside and Teapot frameworks. You can do front-end web with Amber and PharoJS transpiled languages. You can do mobile with Cordova/PhoneGap. You can do data science with PolyMath library, Roassal data visualization, and Moose data analytics platform. You can do machine learning with TensorFlow and Keras. You can do IoT with PharoThings. You can do robotics with PhaROS. You can do virtual reality. You can even script the Unreal game engine!

Roassal demonstration
virtual reality demonstration
Unreal game engine demonstration

In other words, Smalltalk provides a huge stockpile of ammunition. You can fire salvo after salvo after salvo.

What other language can be as easily evangelized? Clojure? Crystal? Dart? Elixir? Golang? Haskell? Julia? Kotlin? Nim? Racket? Rust? TypeScript? Nope. Not even close.

(Previously published at ITNEXT.)

Create your website with WordPress.com
Get started