Editor support for writing assembly and BASIC

[ralphb]

I've added some editor support to the xdt99 tools that assist developers with writing assembly and (Extended) BASIC programs. Right now this includes an extension for the GNU Emacs editor and a plugin for the IntelliJ IDEA IDE. Both editors are available for free and run on Linux, OS X, and Windows.

 

I realize that a full-blown IDE might be somewhat heavy for developing simple TI 99 programs. But I've been using both tools anyway, and their capabilities translate nicely to TI assembly and BASIC. The editor support is independent of the other xdt99 tools and can be used without those.

 

The Emacs extension offers simple syntax highlighting and smart indenting and text formatting. Having the mnemonic field start at the incorrect position drove me nuts, so I decided to do something about editors in the first place.

 

The IDEA plugin offers advanced syntax highlighting, semantic navigation, semantic inline renaming, and usage lists. For example, in BASIC you can "rename" line numbers (a la RESEQUENCE) without changing integer constants, or rename all occurrences of variable "VAR" without affecting variable "VAR1" etc. For assembly sources, you can quickly navigate to the definition of a symbol or find all of its usages, including those in other files.

 

The plugins might still be somewhat rough around the edges, given the limited number of sources I could test them with. I've attached a ZIP file for your convenience, eventually they'll be available from the xdt99 homepage as well.

 

Given that most people feel rather strongly about "their" editor the addressable market here is probably even smaller than usual. But if you think that you spend too much time typing stuff you may find using IDEA useful.

 

 

xdt99-editor-1.0.0.zip

 

[RXB]

Assembly and XB and GPL cool.

[Opry99er]

Wow... This just may replace Notepad++ on my desktop!

 

Very cool!!!

[sometimes99er]

Wow !!! I have to check this out in detail when I get the time. Sounds and looks absolutely wonderful !

[Asmusr]

This is absolutely wonderful, especially since I'm using IntelliJ IDEA for my daily work coding Java and JavaScript. I have been dreaming of something like this for years.

Unfortunately it doesn't work for me right away. After installing the plugin .a99 files get a new icon, so seems to be correctly installed. But I get this error the first time I try to open one of the files:

19:42:19 NoClassDefFoundError: com/intellij/lang/LightPsiParser: com/intellij/lang/LightPsiParser

The following times I get this error instead:

19:42:23 NoClassDefFoundError: net/endlos/xdt99/xas99/parser/Xas99Parser: net/endlos/xdt99/xas99/parser/Xas99Parser

I'm using IntelliJ IDEA 13.1 Ultimate, JRE 1.0.7_40 x86, Windows 7.

Any suggestions?

Thanks,

Rasmus

[Asmusr]

I installed the community edition of IDEA 14, and here the plugin is working fine, so I guess version 13 is missing some classes required by the plugin.

[Asmusr]

Here is the source code for Jet Set Willy if you need some assembly code for testing. I find that jumping from a BL @LABEL instruction to the declaration of LABEL (using Ctrl-B) doesn't work in most cases, without any clear pattern as to when it does work.

source-final-bugfix-release.zip

[ralphb]

Hello Rasmus:

 

19:42:19 NoClassDefFoundError: com/intellij/lang/LightPsiParser: com/intellij/lang/LightPsiParser

I'm using IntelliJ IDEA 13.1 Ultimate, JRE 1.0.7_40 x86, Windows 7.

 

Oh, what a bummer! I can confirm that it's not working on IDEA 13 ... The latest Grammar-Kit, which is used to generate the parser from the BNF, introduces a dependency on class LightPsiParser, which seems to be missing in IDEA 13.

 

I'll have to downgrade my version of Grammar-Kit to generate a plugin that works with both versions. In the meantime I created a preview version for IDEA 13 that seems to work fine (although I did have to disable the custom tab stops).

 

Any feedback is highly welcome, including bug reports such as this one.

 

Cheers

Ralph

 

[Argh, fighting with the attachment feature of AA ...]

[ralphb]

OK, still won't attach my ZIP file, but I'm glad you got it working for now.

 

Edited August 10, 2015 by ralphb

[ralphb]

Here is the source code for Jet Set Willy if you need some assembly code for testing. I find that jumping from a BL @LABEL instruction to the declaration of LABEL (using Ctrl-B) doesn't work in most cases, without any clear pattern as to when it does work.

 

Thanks, that is helpful, and the behavior thus easy to explain: If the source code contains lexical errors, the parser often just stops, or at least re-syncs very badly. I'll probably have to tune the grammar to account for that.

 

And regarding the syntax: Well, there's more than one ... I opted for relaxed whitespace handling when I wrote xas99, and the plugin honors that syntax. One consequence is that the comment field must be separated by two blanks, where \t counts as one. Some of your comments are spaced too closely, raising a syntax error with the plugin. Another source of "errors" are the preprocessor macros IF, ELSE etc., which I call .ifdef, .endif ... in xas99.

 

I'll definitely have to adopt the grammar to match existing real-world coding more closely ...

 

Ralph

[Asmusr]

It works great on another project without any IF/ELSE/ENDIF. I really need to switch from WinAsm99 to using your tools one of these days.

 

I love the ability to safely rename variables. Thanks again, this is really awesome.

[Willsy]

Okay, it looks like I'm going to have to switch to use Ralph's tools. These are too good to ignore. Could I be retiring ASM99 after 10 years? Oh my...!

 

Say, Ralph, how difficult would it be to produce a Forth plugin for the IDEA editor? The cool think about Forth is that it has no syntax to speak of. Each "word" (separated by white space) does something. There are no open parenthesis with comma delimited parameters etc like in C or Java. The concept of syntax doesn't really exist. All I would need for a Forth plugin would be to identify a word, and sets its colour depending on the "family" of the the word (e.g. stack manipulation word, math word, comparison word, logical etc).

 

Are there any links that provide guidance on writing plugins for this editor?

 

Many thanks

[ralphb]

Say, Ralph, how difficult would it be to produce a Forth plugin for the IDEA editor? The cool think about Forth is that it has no syntax to speak of. Each "word" (separated by white space) does something. There are no open parenthesis with comma delimited parameters etc like in C or Java. The concept of syntax doesn't really exist. All I would need for a Forth plugin would be to identify a word, and sets its colour depending on the "family" of the the word (e.g. stack manipulation word, math word, comparison word, logical etc).

 

Willsy, I'm glad you like it.

I couldn't find a Forth plugin for IntelliJ, but it should be relatively simple to come up with one. The biggest task is to write a grammar file for it, like this one.

It's been a while since I did Forth programming (on a TI, of course!), so I don't remember if functions like dup are built-in, or if they're just part of a standard library that is preloaded. In any case, you can basically decide for yourself which functions you want to recognize and assign one or more classes (i.e., colors) to them. All other functions or words would just be vanilla identifiers.

But there are bits and pieces of syntax as well, e.g., function definitions. Those would link your identifiers to each other so that inline renaming would work, more of less automatically.

Feel free to give it a try based on xas99.bnf; you won't have to compile it or anything. I'd be happy to add a Forth mode to the plugin.

BTW, Emacs does have a Forth mode for GNU Forth, which also does code formatting, but Emacs is ... special.

[carlsson]

Out of curiosity, have you considered adding compile functions to your Emacs mode, so you can run command line compiler and assembler tools from within Emacs? I think that is perhaps even more nice than indentation and syntax highlighting. A couple of weeks ago I cobbled together an IntyBASIC mode for Emacs, so if you're interested in extending your mode you can steal some ideas from me, or perhaps you're talented enough in Emacs Lisp to get it working on your own.

[+OLD CS1]

Wow! That looks like an IDE I can sing my teeth into.

[ralphb]

Say, Ralph, how difficult would it be to produce a Forth plugin for the IDEA editor?

 

Well, I've had a look at your TurboForth site (nice!) and cobbled together a grammar plus lexer. Does this look like Forth to you, especially the lexer?

 

For the grammar I'll have to clean up the whitespace handling (that includes assembly and BASIC) before I dare putting out a new plugin version.

 

TForth.zip

[Willsy]

 

Well, I've had a look at your TurboForth site (nice!) and cobbled together a grammar plus lexer. Does this look like Forth to you, especially the lexer?

 

For the grammar I'll have to clean up the whitespace handling (that includes assembly and BASIC) before I dare putting out a new plugin version.

 

TForth.zip

Whoa! Fantastic! Yes this looks very promising indeed. I'll have a look at this tomorrow (Thursday) as I'm out playing guitar tonight. I'll add the other families of words to the files for you. We'll also get Lee to add any words from fbForth, too if you don't mind.

 

You're quite right... There is a little bit of syntax. Here's the rules:

 

"Words" (in other languages called "functions") are defined as follows:

• Shall be preceded by a colon and one or more whitespace (including tab)
• Shall include a name
• Shall include 0 or more references to (pre-existing) words already defined, each separated by one or more whitespace characters or a new line.
• Shall be terminated with a semi-colon

Examples:

: my-word ( some comment)
  some-word another-word
  yet-another word
;
 
: my-word ( some comment ) som-word another-word yet-another-word ;

In-line Comments

• A comment shall begin with the word (
• Note that ( is a word, and therefore may have one or more whitespace characters either side of it
• A comment is terminated with a closing ) character (not a word)
• A comment may include one or more ( characters
• A comment may not inlude a ) character (since the ) character terminates the comment)
• If end-of-line is reached without a terminating ) character, the comment is assumed closed

Examples:

: some-word ( this is an in-line comment) some more code ;
( here is another in line comment, all by itself )
(      here is an other      )
(this is not a comment - there is no space after the open ( character)
( this is a comment the dup at the end is valid even though there is no trailing space)dup
( normally it would be written like this ) dup ( i.e. it would have a space after the closing bracket, but both are valid )

Line Comments

• Line comments may begin anywhere on a line.
• The begin with the word \ which may have one or more white space characters before and after it
• When a line comment is detected, everything up to the end of the line is ignored

Examples:

\ this is a comment
   \    this is a comment
3 constant LIVES \ define a constant for the number of lives
\this is not a comment (there is no such word as \this)
\ \\\\\ this \\ is \ a valid \ comment (because the entire line is ignored)

Strings:

• The words S" and ." define strings.
• A string is terminated with a closing " character
• The " character may not appear in the string
• Whitespace between the opening S" word and the string are included in the string

Examples:

S" This is a string" ( note space between the WORD S" and the start of the string - at least one space required (to identify the word S")
S"      This is also a string"
." This is a string"
."       This is also a string"

I think that's about it. The secret is to know that words and numbers MUST be separated by at least one whitespace character. That's how Forth identifies them:

 

SWAP DUP DROP - 3 separate words - all valid

SwapDupDrop - the system would look for a word called SwapDupDrop - if it's defined, it's valid.

 

Also, a word can have any name (except for a blank space):

: '@@#@]][-=*&^% some-code ;

This is perfectly valid.

 

So is this (if you are a masochist):

: 1 99 ;

Now, 1 + 1 gives you 108. This is because Forth looks for words in its dictionary first. If it doesn't find a matching word, only then does it look to see if it's a valid number.

 

:thumbsup:

Edited August 12, 2015 by Willsy

[Willsy]

Here's the updated TForth.flex file with the updated word families. TForth.zip

 

Hopefully my earlier post makes sense with respect to syntax.

 

Hopefully we can do neat things like folding of definitions, like in the attached screen shots:

 

 

Another question: Is it possible for the editor to make associations of one word with another word?

 

Here's an example:

 

VARIABLE FRED

 

This defines a variable called FRED. It would be VERY cool if the editor could know that since FRED is born from VARIABLE, that FRED is a variable, thus colouring it the same colour as the word VARIABLE.

 

Same for CONSTANT:

 

100 CONSTANT HUNDRED

 

Here, HUNDRED is a child of CONSTANT.

 

Think classes and class instances in java and .net and the like, where the editor knows the parent class of an instance.

 

I think I'm getting a bit carried away

 

Regards

 

Mark

 

[+Lee Stewart]

Whoa! Fantastic! Yes this looks very promising indeed. I'll have a look at this tomorrow (Thursday) as I'm out playing guitar tonight. I'll add the other families of words to the files for you. We'll also get Lee to add any words from fbForth, too if you don't mind.

 

I'd be happy to do that.

: my-word ( some comment ) som-word another-word yet-another-word ;

In-line Comments

• A comment shall begin with the word (
• Note that ( is a word, and therefore may have one or more whitespace characters either side of it
• A comment is terminated with a closing ) character (not a word)
• A comment may include one or more ( characters
• A comment may not inlude a ) character (since the ) character terminates the comment)
• If end-of-line is reached without a terminating ) character, the comment is assumed close

 

Only in terminal mode is end-of-line a terminating character in fbForth (or TI Forth, for that matter). When loading blocks, end-of-line does not exist. The block is one continuous stream of 1024 characters. Same thing goes for ." and any other word that is processed in a similar manner.

 

Also, a word can have any name (except for a blank space):

: '@@#@]][-=*&^% some-code ;

This is perfectly valid.

So is this (if you are a masochist):

: 1 99 ;

Now, 1 + 1 gives you 108. This is because Forth looks for words in its dictionary first. If it doesn't find a matching word, only then does it look to see if it's a valid number.

 

Uh...well...actually, that would be 198.

 

...lee

[Willsy]

Only in terminal mode is end-of-line a terminating character in fbForth (or TI Forth, for that matter). When loading blocks, end-of-line does not exist. The block is one continuous stream of 1024 characters. Same thing goes for ." and any other word that is processed in a similar manner.

Noted (and agreed - same in TF)

 

Uh...well...actually, that would be 198.

 

...lee

Smart arse

[ralphb]

: 1 99 ;

Now, 1 + 1 gives you 108.

 

Uh...well...actually, that would be 198.

 

Actually, wouldn't 1 + 1 be a stack underflow?

 

Got your file, and thanks for the explanations! I'll integrate it into the plugin within the next few days. Today is reserved for the Perseids.

[+Lee Stewart]

Actually, wouldn't 1 + 1 be a stack underflow?

 

Good Forth thinking! It does cause an underflow; but, it doesn't throw an error because + does not check. It simply does the addition of ‘1’ and the contents of one cell below the bottom of the stack, puts the result there and places the next ‘1’ on the stack and shows the stack depth to be only one cell!

 

...lee

[Willsy]

 

Good Forth thinking! It does cause an underflow; but, it doesn't throw an error because + does not check. It simply does the addition of ‘1’ and the contents of one cell below the bottom of the stack, puts the result there and places the next ‘1’ on the stack and shows the stack depth to be only one cell!

 

...lee

See? I'm a genius!

[ralphb]

Alright, here's a much improved version of the plugin for assembly and Extended BASIC. Navigation and refactoring now works for sources with errors, as it should. Error reporting has been made more meaningful, and the assembly mode now handles unknown mnemonics gracefully.

 

The plugin should work in IDEA 13 and IDEA 14, although the tab key doesn't work in IDEA 13 for now. (And if you're getting weird code suggestions when typing, say, the B instruction you probably have some unrelated live templates switched on.)

The TurboForth mode is forthcoming (no pun intended) in a few days.

xdt99-idea.zip

[Asmusr]

Thanks, works well in IDEA 13.