Sugar Programming Language

My journey into language development has been a long and winding one, and Sugar is the culmination of that journey. It all started with Salt, my first attempt at a programming language. I was young, ambitious, and maybe a little naive. I decided to write it in C++, and I was determined to build everything from scratch. I wrote my own BNF grammar, and I was so proud of it. But when it came to actually parsing the code and building the AST, I was in over my head. I spent weeks trying to debug my parser, but I could never get it to work reliably. I particularly struggled to create a reliable recursive descent parser for the BNF format I had designed, often ending up with a half-baked solution that couldn't cope with complex structures. Salt was a failure, but it was a valuable one. It taught me that language development is hard, and that I had a lot to learn about robust parsing and Abstract Syntax Trees (ASTs).

After the Salt debacle, I took a step back and decided to try a different approach. I created Pepper, a much simpler language written in Python. Pepper was an interpreted language, which meant that it was executed line by line, without a complex parsing and compilation step. This made it much easier to implement, and I was able to get a working version up and running in a relatively short amount of time. Crucially, due to my struggles with Salt, I completely skipped AST generation for Pepper. You can check out the source code for Pepper on my GitHub. Pepper was a success, but it was also very limited. It was slow, and it lacked many of the features of a modern programming language, such as the ability to handle arbitrary nesting of code blocks, a capability that relies heavily on a robust AST. I knew that I could do better.

And so, I started working on Sugar. I wanted to create a language that had the simplicity of Pepper, but with the power and structure that a clear grammar and AST could provide. I went back to the drawing board, and I designed Sugar from the ground up. I chose to write it in Python, but this time I used the Lark library to handle the parsing. This was a game-changer. Lark is an amazing tool that makes it easy to create powerful and efficient parsers. With Lark, I was able to create a robust parser for Sugar in a fraction of the time it would have taken me to write one from scratch. Lark was particularly helpful by warning me about ambiguous grammar rules as I defined them, a major hurdle I faced with Salt. Moreover, Lark allowed me to parse non-terminal statements, like a guard_clause, directly into dedicated Python classes such as a GuardClause object in my transformer before interpretation. This approach made my interpreter much easier to build, as I only had to implement visit methods for each class returned by Lark and my transformer, instead of trying to parse and interpret each statement all at once. This structured approach was fundamental to Sugar's ability to handle arbitrary nesting and its advanced features.

The Power of Sugar

Sugar is a statically-typed, object-oriented, interpreted language with a focus on readability and simplicity. I've always enjoyed static typing, and despite Sugar being interpreted, I chose this approach as it makes writing Sugar code a much easier experience for developers. If a developer tries to assign something incompatible to a variable, the type checker immediately catches the error. This prevents a whole class of bugs that would typically appear as runtime errors in dynamically typed languages, leading to fewer surprises and more predictable code. My negative experience with Salt's dynamically typed nature, where trying to find a robust way to store an environment of random types in C++ became "horrible," strongly influenced this decision. While it made the language design a bit more complex initially, Sugar's type checker is now incredibly robust, meticulously checking function return statements and their types, all assignments, type conversions, and arguments passed to typed parameters. This strong type system also uniquely allows for advanced features like function overloading, where functions with differently typed parameters can share the same name, similar to languages like C++ or Java. Sugar also supports custom types, which are only possible to implement reliably with such a robust type system.

• A powerful and expressive static type system, enabling robust code and custom types.
• First-class functions and closures.
• A full-featured object-oriented system with classes, inheritance, and interfaces.
• A powerful standard library with a wide range of functionality.
• A simple and intuitive syntax.

But the feature that I'm most proud of is Sugar's pattern matching. I was heavily inspired by Rust's match statements, and I wanted to bring that same power and expressiveness to Sugar. With Sugar's pattern matching, you can deconstruct complex data structures with ease. For example, you can match on the types of values in a tuple:

DEF my_tuple #(#int, #str) = (10, "hello")

MATCH my_tuple
    CASE (#int, #str) do
        IO:PRINT:("It's an int and a string!")
    CASE (#int, #int) do
        IO:PRINT:("It's two ints!")
    DEFAULT do
        IO:PRINT:("It's something else.")
end

You can also use guards to add extra conditions to your matches:

DEF my_var #int = 10

MATCH my_var
    CASE n if $n > 5$ do
        IO:PRINT:("It's greater than 5!")
    CASE n if $n < 5$ do
        IO:PRINT:("It's less than 5!")
    DEFAULT do
        IO:PRINT:("It must be 5!")
end

Error Handling: A Significant Challenge

One of the most challenging aspects of building Sugar, and one I really struggled with, was implementing a robust and user-friendly error handling system. My goal was to support familiar constructs like RAISE Error("message") for throwing exceptions, and TRY CATCH e #Error do FINALLY do for handling them. This meant I had to develop a sophisticated mechanism to map Sugar's internal error types and custom exception types directly to Python's underlying exception hierarchy. For example, Sugar's Error type is an alias to Python's Exception, but I also needed to support mapping more specific Sugar errors to their Python counterparts, such as ValueError to Python's ValueError, or KeyError to Python's KeyError:

TYPE MyError EXTENDS Error
    message #str
end

# Example of internal mapping (conceptual):
# base_errors = {
#     "Error": SugarError(Exception),
#     "ValueError": SugarError(ValueError),
#     # ... other mappings
# }

It was particularly tricky to handle raising errors with both custom and built-in Sugar types while ensuring they could be caught correctly by TRY...CATCH blocks, whether they were catching a specific custom type or a more general base error. To resolve this, I eventually developed a _resolve_exception_base_name class. This class's purpose is to accurately resolve the true base exception name for a given declared exception type within Sugar, intelligently handling both direct base errors and custom types that extend them. This involved a lot of intricate logic and checks using isinstance to ensure the correct Python exception was raised or caught, effectively bridging Sugar's type system with Python's exception propagation.

Building the Standard Library

While the standard library's design itself proved to be relatively straightforward for me, it was still a crucial part of building Sugar. I wanted Sugar to be a practical language, and that meant it needed a rich set of built-in functions and modules. I started with the basics, like a Math module with common mathematical functions, and an IO module for input and output. But I didn't stop there. I also added a Time module for working with dates and times, and a Random module for generating random numbers.

The main challenge here was figuring out how to interface with the underlying Python libraries that I was using to implement the standard library. This was a complex task of integrating two different worlds, and it was incredibly satisfying to see it all come together.

The Future of Sugar

I'm incredibly proud of what I've accomplished with Sugar, but I'm not done yet. I have a lot of ideas for how I can make it even better. I want to add a more powerful type system, with support for generics and type inference. I also want to improve the performance of the interpreter, and I'm even considering writing a compiler for Sugar that can target a low-level virtual machine like the JVM or LLVM, which would involve a significant architectural shift from its current interpreted state.

Building Sugar has been an amazing journey. It's taught me so much about programming language design, and it's given me a newfound appreciation for the tools that I use every day. I'm excited to see what the future holds for Sugar, and I can't wait to see what I'll build with it.