_{Jamie Balfour's}Personal blog

ZPE 1.9.1 was officially released today marking the first of the 2021 versions of ZPE. 

ZPE 1.9.1, codenamed Kelso, comes in with a large number of new features and performance improvements and is recommended to all users. 

Version 1.9.1 finally fixes undefined comparisons, improves the LAME parser, adds support for empty associative arrays to the JSON parser, fixes a parameters issue, and fixes negative numbers in the binary calculator by cutting them to the appropriate number of bits (this was tricky to implement but I figured a way to do it as I always do).

But, the big feature is the match construct. This is the first new construct in a while as well (it's actually an expression but since ZPE doesn't have expressions in the documentation and it flows like an inline construct I'm calling it a construct). The match construct is now fully in the documentation and it's a pretty powerful construct. 

Did I mention that ZPE 1.9.1 is actually smaller than ZPE 1.8.13 too?

One of the new features that's been recently added to PHP 8 which I'm also excited about is that of union types. ZPE has actually had union types for a while, but it was never formalised in the YASS language and therefore never actually enabled. In the next few versions, I will bring union types to ZPE. 

There are other plans for ZPE in terms of types actually, since it was suggested that untyped and typed code should be mixable. Thus, in the next few versions, I will bring the ability to mix and match hard typing and soft typing in the code. 

ZPE 1.8.11 is now available and is another small update in terms of the number of features but big in what the new features bring. 

The first big new feature is support for infinite parameters. This is done within the definition with the double dot notation such as function x($y, $z ..). There is more in the documentation.

Reference functions have been completely redesigned, as have objects. With objects, there is a 2x performance increase whilst with reference functions, there is a much larger performance increase as well as a stability improvement. Reference functions can now be chained to each other much easier and this change requires much less code to do it.

ZPE 1.9.1 might be over three months away following the version/year release cycle that I developed for it, but nonetheless, it is a good time for me to explain the new codename system chosen for this cycle of ZPE versions. 

After much debating with my brother over this one, I have decided to name them after places in the Scottish Borders. I thought I'd start off with a few version names:

• 1.9.1 - Kelso
• 1.9.2 - Selkirk
• 1.9.3 - Hawick
• 1.9.4 - Peebles
• 1.9.5 - Melrose
• 1.9.6 - St Boswells
• 1.9.7 - Galashiels
• 1.9.8 - Chirnside
• 1.9.9 - Duns
• 1.9.10 - Lauder
• 1.9.11 - Earlston
• 1.9.12 - Roxburgh
• 1.9.13 - Allanton

There may be more names in the future but that's all of them for now. You may also note that I am no longer starting on minor version 0, e.g. 1.9.0, but starting on version 1, e.g. 1.9.1. This is a change that makes sense to me when developing.

ZPE 1.8.10 is another big improvement with the new security sandbox being improved considerably. It also adds big new features to the ZPEClient and ZPEServer modes that allow SSH-like commands to be executed as well as SFTP-like commands.

I have also added in my own file system management tools to assist in correctly parsing file pathnames such as ../ on Windows systems as well as all other platforms for a consistent file management system.

The new logo has also been added to the GUI and I've improved the debug mode.

Once again, ZPE 1.8.10 is a big update like its predecessors were. It's now available for download from my website in the usual place.

SFTP is an amazing protocol. I've been using it for a good 7 or 8 years now and it's been one of the most useful tools I have ever used.

SFTP features commands like get and put. 

Well ZPE 1.8.7 added the send> command to the ZPEClient which allows sending a file to a ZENServer. Prior to this, I had been ensuring that all data between the client and server is secured using my own encryption system. Well, without hesitation, I knew that this had to come to file transfer too. So now ZPE can send and receive files that are encrypted and decrypted within the client and server. This breakthrough is a compelling reason to use ZPE since it offers a very strong alternative to SFTP. 

Further to this, to make it even more powerful the client now offers the ls (list files locally) command and the rls (remotely list files) commands. 

Past versions of ZPE have often contained flawed features such as the LAMP evaluator or minor things like performance glitches. This is often the case with any project. But any project will have flaws in it somewhere. ZPE is no different.

However, the stability of ZPE has got to the point where I believe that most of the flaws and errors that stop it being usable have been ironed out.

Over the last few months, ZPE 1.8.x has been removing old code and has cut down over 10,000 lines of code from both the runtime and compiler to make it more streamlined but also to improve stability. Further to this, features from very early versions have been revisited and revised to ensure much better performance and reliability - writing a programming language, compiler and runtime is quite tricky you know! Most of the issues actually come from the runtime, since I spent a lot of time in 2016 improving the compiler whereas the runtime just got a small bit of TLC.

Another major issue that often occurs is when new compiler features are added that actually could interfere with another feature. Now I spend a lot of time drawing up the ideas so don't get me wrong, I actually do plan. But sometimes it's difficult to see these potential collisions from the offset. For example, the move to the fat arrow syntax for lambda functions caused all sorts of issues until I decided to merge functions with lambda functions (that in itself caused further issues). More recently I discovered after going through the compiler that the generate_parameters function would allow values to be added in a function declaration's parameters: e.g. function foo ($x, 10) { print($x) } because the generate_parameters function was designed for both formal parameters and actual parameters. This has been fixed by separating these into generate_parameters and generate_arguments.

If you are writing programs using ZPE and compiling them, make sure to recompile them with the latest versions as soon as they are available as byte codes are constantly changing and being added. A new feature I'm adding into the runtime is to check which version compiled the program before attempting to run it since old versions may have different byte codes.

Since ZPE 1.8.8 was released as a fix for a bug in ZPE 1.8.7, ZPE 1.8.9 will be released on 05/08/2020.

ZPE is very flexible and powerful now. But there are still things that I've wanted to add since I started developing it. The compiler is very flexible and it's so easy to add new features that whenever one pops into my head I want to implement it. But sometimes a feature itself become complicated or changes the way the system works currently and I try to find a more standard approach to its implementation.

Two features I speak of are named parameters and infinite parameters. 

First, let's look at infinite parameters. Infinite parameters have been on the cards for a long time and actually existed for a short period of time in ZPE 1.5.x but were simply removed due to complications the features brought. In theory, they should be easy to implement because the syntax for them was something like:

$x, $y ..

On the compiler side, this is easy to implement. But on the runtime side, I wasn't sure how to represent it. Should $y be a ZPEList type? ZPEList would make sense so that $y[0] would be the first variable and $y[1] the second and so on and so forth. Infinite parameters are actually supported by internal native methods by default, but not by defined functions or external native methods added by plugins. This is why I feel this is a priority feature.

The other is named arguments. This is something implemented in ZPE 1.3.x for a period then removed before release. It's coming soon to ZPE (probably late this year) and it's one of my favourite up and coming features that will be released this year. 

Named arguments allow arguments to be specified in an out of order manner by specifying the name. This is roughly how it will look:

function makeperson($name, $age, $favourite_colour)
  print("Your name is " & $name & " you are " & $age & " and your favourite colour is " & $favourite_colour)
end function 
function main($args)
  makeperson (age = 10, name = "Jack", favourite_colour = "Purple") 
end function

Notice that the arguments are out of order and specified by their name. This is the main benefit of this new feature.

I thought I'd share the new logos on my blog. These new logos are concepts I am happy with. 

ZPE (latest) is now also at 24,000 downloads and after running an SQL query for all ZPE versions I got a total of 27,000 downloads!

ZPE 1.8.7 is a broken build, the first in a long time. ZPE 1.8.7 was intending to modularise the software by turning from using duplicated code across Velocity and ZPE into one JAR file that could easily be used by either. Unfortunately, this caused more issues than it fixed. As a result of this, I have simply reverted back to the original code design.

Why was this not noticed?

I will admit that over the last few months my testing has been quite lax and undisciplined. As a result, I simply felt that I didn't need to test this in all platforms (which is stupid since the new modularisation should have meant all platform testing should have been done right away) and as a result only tested on a few devices (by the way, in ZPE 1.8.7 is fully working on JDKs with JavaFX including macOS versions).

ZPE 1.8.8

ZPE 1.8.8 is actually awesome. It now includes its own FTP server within the server mode that means that you can transfer a file to the server from the client. This exciting new feature was going to get more information as it developed but the pressing issue of dealing with 1.8.7's catastrophe meant that I needed to focus on that.

Further to that, ZPE 1.8.8 fixes a long-standing issue of negative exponents and is built straight into the Zenith Parsing Engine itself and is supported by my CSV, JSON and XML parsers too. 

ZPE 1.8.9 will continue to work on the performance improvements ZPE 1.8.8 has laid out as well as continuing the development of ZPEKit.

ZPE has for a long time been able to compile and password protect compiled applications. Now with ZPE comes SecureCode (codenamed Diamond Peak). SecureCode is a built-in part of the ZPE package that secures code using a special algorithm. Code can be decrypted by the engine and then run directly from it. Secure code has been in development for months, only to finally come to fruition now.

This new form of security adds layers of protection to applications that make ZPE even more secure. The built-in decryption engine will be included within the up and coming YASS Executable specification.

In a nutshell, the encryption and decryption algorithms use the password as the initialisation vector but since the password is not stored as plain text and can only be verified by encrypting a users input and comparing it against the encrypted password, there is no way to decrypt the code. Further, the compiler applies the encryption algorithm a number of times to strengthen the security of the file.

function main(){

 print("Hello world")

}

When compiled, the file would like:

^@^Esr^@,jamiebalfour.zpe.core.YASSCompiledExecutable}N^P;<8A>^B^@^GZ^@^LexperimentalJ^@^DtimeL^@^Fauthort^@^RLjava/lang/String;L^@^Pcompiler_versionq^@~^@^AL^@^Dnameq^@~^@^AL^@^Hpasscodeq^@~^@^AL^@^Gprogramt^@^RLjava/lang/Object;xp^@^@^@^@^M^Xbt^@^Njamiebalfour04t^@^G1.8.8.0t^@^@t^@^@sr^@)jamiebalfour.zpe.core.YASSCompiledProgramP^E<9E> ^S<8F>^B^@^C[
^@ functionst^@^][Ljamiebalfour/types/ZPEPair;[^@
structuresq^@~^@        [^@     variablesq^@~^@ xpur^@^][Ljamiebalfour.types.ZPEPair;<82>}%LS^B^@^@xp^@^@^@^Asr^@^Zjamiebalfour.types.ZPEPairu`cӜ^B^@^BL^@^Anq^@~^@^BL^@^Avq^@~^@^Bxpt^@^Dmainsr^@^Zjamiebalfour.zpe.core.FAST^@^@^@^@^@^@^@^A^B^@      B^@^Kreturn_typeB^@^EscopeB^@^DtypeL^@^Mdocumentationq^@~^@^BL^@^Bidq^@~^@^AL^@^Dleftt^@^Ljamiebalfour/zpe/core/FAST;L^@^Fmiddleq^@~^@^QL^@^Dnextq^@~^@^QL^@^Evalueq^@~^@^Bxp^@^@pq^@~^@^Oppsq^@~^@^P^@^@pq^@~^@^Osq^@~^@^P^@^@^Cpt^@^Eprintpppsq^@~^@^P^@^@pq^@~^@^Gpppsq^@~^@^P^@^@^Hpq^@~^@^Gpppt^@^KHello worldppsq^@~^@^P^@^@pq^@~^@^Gpppppuq^@~^@^K^@^@^@^@uq^@~^@
^K^@^@^@^@

But when using SecureCode it looks like:

^@^Esr^@,jamiebalfour.zpe.core.YASSCompiledExecutable}N^P;<8A>^B^@^GZ^@^LexperimentalJ^@^DtimeL^@^Fauthort^@^RLjava/lang/String;L^@^Pcompiler_versionq^@~^@^AL^@^Dnameq^@~^@^AL^@^Hpasscodeq^@~^@^AL^@^Gprogramt^@^RLjava/lang/Object;xp^@^@^@^Oryt^@^Njamiebalfour04t^@^G1.8.8.0t^@^@t^@<$2a$10$gKm.f.P6is/VObd9ZtnBreOj5Lu6fIhJ4P7snMp/VJzgdEL4aHUpiur^@^B[B^W^F^HT^B^@^@xp^@^@^Bp<99>^D(<84>E^G^?^?.MTy<8C>1c^LESCD<89>L"$= BB^Y<8C>^K#^T^F^N<85> gs<9E>=F   ESC-/^NԶ7^C|'^]<9F>^EҏwAQc3F<9B>K
<84>^F<91>|'S<9F>^X+^^MY^O^Bډ^GAͿ_^D<9A>&^E<91>U*<98>o"s^Tƭӏܙ-Jt^Y.^V<90>R^E^ZiZY^N<9D>s<8B>Ʋ^U^TRi^?Vi<96>=c^NnɄiz^Xe-dr^$)<9A>*^X    <0^V<87>N^?^DfT,n<95>k1-<90>*^L^VESC^_<8E>6^P<9D>u<Ls
^O^Lio^NM<9D>^B <83><9B><83> '<88>^V^W^Q8ESC}m5q<91>Wp~
<98>Q<9B>~<81><8A>Uo%,^V+g}<81><96>+<89>^V<95>^T,4+@qVESC]
^_<8C>˼<90>ђo<U+0083>e
{<91>v<93>w虁^Fn6E^BN       ȱW^G^V^@3<8C>q^YMQH<80>.
3Ż<8B>Gc3􃍜B<9A>^MSOO^?P<9A>^R^@<87>^_<83>7q^RIF"<80>g@3zEhp^ m*<80>#,ESCzܘ
Í“v<97><8C><82><9B>j^]N<91>b|j+E^N<9C>Ê¥RA8RTÖº<94>I[C^]<92>-
%<81>^P^H<90>mt        K_*a<87><9D><91>Y^Z#PD^]:^L<96><84>'%Yao^SÓ·Q^U
<84>B]^]ż<8A>c^H8^E^AlNwn9C<81>bnm2<93>=?
Gg

Security and safety have always been paramount within ZPE since the release of version 1.5 with last year's version 1.7.x making the server and client system even more secure using a special public and private key encryption method. ZPE 1.8.x aims to bring further security features to the package in due time. SecureCode is expected later this month.