_{Jamie Balfour's}Personal blog

Past versions of ZPE have often contained flawed features such as the LAMP evaluator or minor things like performance glitches. This is often the case with any project. But any project will have flaws in it somewhere. ZPE is no different.

However, the stability of ZPE has got to the point where I believe that most of the flaws and errors that stop it being usable have been ironed out.

Over the last few months, ZPE 1.8.x has been removing old code and has cut down over 10,000 lines of code from both the runtime and compiler to make it more streamlined but also to improve stability. Further to this, features from very early versions have been revisited and revised to ensure much better performance and reliability - writing a programming language, compiler and runtime is quite tricky you know! Most of the issues actually come from the runtime, since I spent a lot of time in 2016 improving the compiler whereas the runtime just got a small bit of TLC.

Another major issue that often occurs is when new compiler features are added that actually could interfere with another feature. Now I spend a lot of time drawing up the ideas so don't get me wrong, I actually do plan. But sometimes it's difficult to see these potential collisions from the offset. For example, the move to the fat arrow syntax for lambda functions caused all sorts of issues until I decided to merge functions with lambda functions (that in itself caused further issues). More recently I discovered after going through the compiler that the generate_parameters function would allow values to be added in a function declaration's parameters: e.g. function foo ($x, 10) { print($x) } because the generate_parameters function was designed for both formal parameters and actual parameters. This has been fixed by separating these into generate_parameters and generate_arguments.

If you are writing programs using ZPE and compiling them, make sure to recompile them with the latest versions as soon as they are available as byte codes are constantly changing and being added. A new feature I'm adding into the runtime is to check which version compiled the program before attempting to run it since old versions may have different byte codes.

Since ZPE 1.8.8 was released as a fix for a bug in ZPE 1.8.7, ZPE 1.8.9 will be released on 05/08/2020.

ZPE is very flexible and powerful now. But there are still things that I've wanted to add since I started developing it. The compiler is very flexible and it's so easy to add new features that whenever one pops into my head I want to implement it. But sometimes a feature itself become complicated or changes the way the system works currently and I try to find a more standard approach to its implementation.

Two features I speak of are named parameters and infinite parameters. 

First, let's look at infinite parameters. Infinite parameters have been on the cards for a long time and actually existed for a short period of time in ZPE 1.5.x but were simply removed due to complications the features brought. In theory, they should be easy to implement because the syntax for them was something like:

$x, $y ..

On the compiler side, this is easy to implement. But on the runtime side, I wasn't sure how to represent it. Should $y be a ZPEList type? ZPEList would make sense so that $y[0] would be the first variable and $y[1] the second and so on and so forth. Infinite parameters are actually supported by internal native methods by default, but not by defined functions or external native methods added by plugins. This is why I feel this is a priority feature.

The other is named arguments. This is something implemented in ZPE 1.3.x for a period then removed before release. It's coming soon to ZPE (probably late this year) and it's one of my favourite up and coming features that will be released this year. 

Named arguments allow arguments to be specified in an out of order manner by specifying the name. This is roughly how it will look:

function makeperson($name, $age, $favourite_colour)
  print("Your name is " & $name & " you are " & $age & " and your favourite colour is " & $favourite_colour)
end function 
function main($args)
  makeperson (age = 10, name = "Jack", favourite_colour = "Purple") 
end function

Notice that the arguments are out of order and specified by their name. This is the main benefit of this new feature.

I thought I'd share the new logos on my blog. These new logos are concepts I am happy with. 

ZPE (latest) is now also at 24,000 downloads and after running an SQL query for all ZPE versions I got a total of 27,000 downloads!

ZPE 1.8.7 is a broken build, the first in a long time. ZPE 1.8.7 was intending to modularise the software by turning from using duplicated code across Velocity and ZPE into one JAR file that could easily be used by either. Unfortunately, this caused more issues than it fixed. As a result of this, I have simply reverted back to the original code design.

Why was this not noticed?

I will admit that over the last few months my testing has been quite lax and undisciplined. As a result, I simply felt that I didn't need to test this in all platforms (which is stupid since the new modularisation should have meant all platform testing should have been done right away) and as a result only tested on a few devices (by the way, in ZPE 1.8.7 is fully working on JDKs with JavaFX including macOS versions).

ZPE 1.8.8

ZPE 1.8.8 is actually awesome. It now includes its own FTP server within the server mode that means that you can transfer a file to the server from the client. This exciting new feature was going to get more information as it developed but the pressing issue of dealing with 1.8.7's catastrophe meant that I needed to focus on that.

Further to that, ZPE 1.8.8 fixes a long-standing issue of negative exponents and is built straight into the Zenith Parsing Engine itself and is supported by my CSV, JSON and XML parsers too. 

ZPE 1.8.9 will continue to work on the performance improvements ZPE 1.8.8 has laid out as well as continuing the development of ZPEKit.

ZPE has for a long time been able to compile and password protect compiled applications. Now with ZPE comes SecureCode (codenamed Diamond Peak). SecureCode is a built-in part of the ZPE package that secures code using a special algorithm. Code can be decrypted by the engine and then run directly from it. Secure code has been in development for months, only to finally come to fruition now.

This new form of security adds layers of protection to applications that make ZPE even more secure. The built-in decryption engine will be included within the up and coming YASS Executable specification.

In a nutshell, the encryption and decryption algorithms use the password as the initialisation vector but since the password is not stored as plain text and can only be verified by encrypting a users input and comparing it against the encrypted password, there is no way to decrypt the code. Further, the compiler applies the encryption algorithm a number of times to strengthen the security of the file.

function main(){

 print("Hello world")

}

When compiled, the file would like:

^@^Esr^@,jamiebalfour.zpe.core.YASSCompiledExecutable}N^P;<8A>^B^@^GZ^@^LexperimentalJ^@^DtimeL^@^Fauthort^@^RLjava/lang/String;L^@^Pcompiler_versionq^@~^@^AL^@^Dnameq^@~^@^AL^@^Hpasscodeq^@~^@^AL^@^Gprogramt^@^RLjava/lang/Object;xp^@^@^@^@^M^Xbt^@^Njamiebalfour04t^@^G1.8.8.0t^@^@t^@^@sr^@)jamiebalfour.zpe.core.YASSCompiledProgramP^E<9E> ^S<8F>^B^@^C[
^@ functionst^@^][Ljamiebalfour/types/ZPEPair;[^@
structuresq^@~^@        [^@     variablesq^@~^@ xpur^@^][Ljamiebalfour.types.ZPEPair;<82>}%LS^B^@^@xp^@^@^@^Asr^@^Zjamiebalfour.types.ZPEPairu`cӜ^B^@^BL^@^Anq^@~^@^BL^@^Avq^@~^@^Bxpt^@^Dmainsr^@^Zjamiebalfour.zpe.core.FAST^@^@^@^@^@^@^@^A^B^@      B^@^Kreturn_typeB^@^EscopeB^@^DtypeL^@^Mdocumentationq^@~^@^BL^@^Bidq^@~^@^AL^@^Dleftt^@^Ljamiebalfour/zpe/core/FAST;L^@^Fmiddleq^@~^@^QL^@^Dnextq^@~^@^QL^@^Evalueq^@~^@^Bxp^@^@pq^@~^@^Oppsq^@~^@^P^@^@pq^@~^@^Osq^@~^@^P^@^@^Cpt^@^Eprintpppsq^@~^@^P^@^@pq^@~^@^Gpppsq^@~^@^P^@^@^Hpq^@~^@^Gpppt^@^KHello worldppsq^@~^@^P^@^@pq^@~^@^Gpppppuq^@~^@^K^@^@^@^@uq^@~^@
^K^@^@^@^@

But when using SecureCode it looks like:

^@^Esr^@,jamiebalfour.zpe.core.YASSCompiledExecutable}N^P;<8A>^B^@^GZ^@^LexperimentalJ^@^DtimeL^@^Fauthort^@^RLjava/lang/String;L^@^Pcompiler_versionq^@~^@^AL^@^Dnameq^@~^@^AL^@^Hpasscodeq^@~^@^AL^@^Gprogramt^@^RLjava/lang/Object;xp^@^@^@^Oryt^@^Njamiebalfour04t^@^G1.8.8.0t^@^@t^@<$2a$10$gKm.f.P6is/VObd9ZtnBreOj5Lu6fIhJ4P7snMp/VJzgdEL4aHUpiur^@^B[B^W^F^HT^B^@^@xp^@^@^Bp<99>^D(<84>E^G^?^?.MTy<8C>1c^LESCD<89>L"$= BB^Y<8C>^K#^T^F^N<85> gs<9E>=F   ESC-/^NԶ7^C|'^]<9F>^EҏwAQc3F<9B>K
<84>^F<91>|'S<9F>^X+^^MY^O^Bډ^GAͿ_^D<9A>&^E<91>U*<98>o"s^Tƭӏܙ-Jt^Y.^V<90>R^E^ZiZY^N<9D>s<8B>Ʋ^U^TRi^?Vi<96>=c^NnɄiz^Xe-dr^$)<9A>*^X    <0^V<87>N^?^DfT,n<95>k1-<90>*^L^VESC^_<8E>6^P<9D>u<Ls
^O^Lio^NM<9D>^B <83><9B><83> '<88>^V^W^Q8ESC}m5q<91>Wp~
<98>Q<9B>~<81><8A>Uo%,^V+g}<81><96>+<89>^V<95>^T,4+@qVESC]
^_<8C>˼<90>ђo<U+0083>e
{<91>v<93>w虁^Fn6E^BN       ȱW^G^V^@3<8C>q^YMQH<80>.
3Ż<8B>Gc3􃍜B<9A>^MSOO^?P<9A>^R^@<87>^_<83>7q^RIF"<80>g@3zEhp^ m*<80>#,ESCzܘ
Í“v<97><8C><82><9B>j^]N<91>b|j+E^N<9C>Ê¥RA8RTÖº<94>I[C^]<92>-
%<81>^P^H<90>mt        K_*a<87><9D><91>Y^Z#PD^]:^L<96><84>'%Yao^SÓ·Q^U
<84>B]^]ż<8A>c^H8^E^AlNwn9C<81>bnm2<93>=?
Gg

Security and safety have always been paramount within ZPE since the release of version 1.5 with last year's version 1.7.x making the server and client system even more secure using a special public and private key encryption method. ZPE 1.8.x aims to bring further security features to the package in due time. SecureCode is expected later this month.

Today I am happy to announce the release of ZPE 1.8.7 known by its codename as Portman.

Portman brings many minor updates plus better support for my new Velocity Web Server. One of the most significant changes to ZPE is the removal of the ZPE prefix in front of internal object definitions. For example, the ZPEHTMLBuilder is now just the HTMLBuilder and the ZPEUI becomes UIBuilder. Speaking of objects, there is now a new Calculator object which uses a compiler to compile expressions into fast-to-execute mathematical formulae. 

Further to that, ZPE 1.8.7 introduces ZPEKit (pronounced zippy kit), a new API for accessing the compiler, interpreter and runtime from a single interface. The introduction of this so far only accesses the compiler, but within the next few versions, ZPEKit will become the only way to access the compiler, interpreter and runtime. 

ZPE 1.8.6 deprecated the ZPEWebServer and I expect ZPE 1.8.9 (James) will remove the ZPEWebServer entirely.

I'm happy to announce that ZPE 1.8.7 will introduce ZPEKit. ZPEKit is a complete redesign of the infrastructure of the ZPE application and will make it even easier than before to access the functions that you might need as a developer. 

The plugin interface will also be more accessible and I plan to bring more native function capabilities to the front end of the application. 

Further to this, I am now developing an image editing program in Java that will use ZPE as the core of the application, so this move is really important to make it easier for me to develop this application.

ZPE version 1.8.6 (Younis) is a big update. 

One of the biggest changes to it is the deprecation of the built-in web server. This has been deprecated in favour of a much more streamlined way of working.

The reason for this. Well, there are a few reasons. The first is that it contradicts the Velocity Web Server (VWS) that I'm building. The ZPE web server was always slow compared with my new VWS which is designed to be compatible with a variety of languages and environments. VWS works with ZPE through what's called a Velocity Module hook now anyway. 

For now the server remains in place but in the next few weeks I plan to remove it altogether and perhaps reimplement it using the basis of VWS.

I was reading through parts of my blog looking for one specific post and came across one of my favourite posts on my website. I'm referring to this one here. This post discussed a form of code optimisation that can be carried out before compilation in some languages including ZPE/YASS.

Specifically, this post discussed performance differences between using variables where a length value (in this case the number of items in a list) is stored in a variable versus inline function calls to check the size of that list each iteration of the loop. It is understandable that the former would have lower latency and indeed lower memory consumption. 

After reading the post again, I thought I'd try it with the newest version of ZPE. At first, my thoughts were it was going to be slower because of the compilation time being increased to allow for compiler optimisation that the new ZPE offers. I was wrong. 

ZPE 1.8.5 is considerably faster on both for loops provided in the example, performing only slightly better in the user optimised version. 

Results for the first for loop in ZPE 1.8.5 were:

real 0m0.454s
user 0m1.016s
sys 0m0.118s

Compared with 2016's 1.4.2E, which got:

real 0m2.071s
user 0m2.914s
sys 0m0.455s

It's taken a while for me to finally get here, but ZPE is finally adding further optimisations for mathematical and logical operations. 

For logic, one optimisation that has been highlighted as a potential area for improvement is the use of nested if statements and optimising them with else ifs instead.

For mathematics, it will include an optimisation that compiles static operations such as 1 + 5, but it will also look to improve the performance through the use of variables within mathematics. 

Since ZPE has begun to include optimisations since ZPE 1.7.8, it has been a major focus and major shift to improve the performance of compiled scripts. I am also looking to improve the performance of web-based scripts and adding in the ability to compile them too.

LAME2 is also under development, designed to improve performance of LAME which itself improved performance over LAMP this time last year. LAME2 will also embed its own optimisations. 

Also, I thought I'd take this opportunity to inform you of my recent use cases for ZPE since it now responds to webhooks much more efficiently. My most recent ZPE project has been on the home Raspberry Pi, which is set up to respond to web requests and manage them accordingly. We are now using ZPE within our smart home setup and I'm very happy to show people how it works. If you are interested, please contact me via email.