dbidwell/stationeers_lang
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Compare commits

base: dbidwell:0.1.1
Branches Tags
dbidwell:master dbidwell:indexing
dbidwell:0.4.0 dbidwell:0.3.4 dbidwell:0.3.3 dbidwell:0.3.2 dbidwell:0.3.1 dbidwell:0.3.0 dbidwell:0.2.4 dbidwell:0.2.3 dbidwell:0.2.2 dbidwell:0.2.1 dbidwell:0.2.0 dbidwell:0.1.2 dbidwell:0.1.1
...
compare: dbidwell:0.2.3
Branches Tags
dbidwell:indexing dbidwell:master
dbidwell:0.4.0 dbidwell:0.3.4 dbidwell:0.3.3 dbidwell:0.3.2 dbidwell:0.3.1 dbidwell:0.3.0 dbidwell:0.2.4 dbidwell:0.2.3 dbidwell:0.2.2 dbidwell:0.2.1 dbidwell:0.2.0 dbidwell:0.1.2 dbidwell:0.1.1

31 Commits
0.1.1 ... 0.2.3

Author SHA1 Message Date
Devin Bidwell
2b26d0d278 Update changelog 2025-12-11 02:26:20 -07:00
Devin Bidwell
236b50c813 Allow syscalls in infix operations 2025-12-11 02:24:01 -07:00
Devin Bidwell
342b1ab107 Fix function invocation stack underflow 2025-12-11 01:03:43 -07:00
Devin Bidwell
0732f68bcf Merge pull request #24 from dbidwell94/documentation 
QOL
 2025-12-10 18:01:21 -07:00
Devin Bidwell
0ac010ef8f Fixed documentation rendering and added ternary expressions 2025-12-10 18:00:20 -07:00
Devin Bidwell
c2208fbb15 Fixed some formatting issues with header markdowns for Stationpedia 2025-12-10 13:39:58 -07:00
Devin Bidwell
295f062797 Merge pull request #23 from dbidwell94/slot-logic 
Slot logic
 2025-12-10 00:11:45 -07:00
Devin Bidwell
9c260ef2d5 Fixed bug where infix wouldn't rewind when encountering a comma, causing the rest of a syscall parse in an assignment expression to fail 2025-12-10 00:09:16 -07:00
Devin Bidwell
0fde11a2bf Added support for syscalls with assignment expressions 2025-12-09 23:57:19 -07:00
Devin Bidwell
b21d6cc73e Found bug, unable to do an assignment expression with a syscall 2025-12-09 23:45:10 -07:00
Devin Bidwell
f19801d4e6 Merge pull request #22 from dbidwell94/logos 
Tech Debt Cleanup
 2025-12-09 17:41:02 -07:00
Devin Bidwell
46500a456a Add support for colorized comments 2025-12-09 17:38:35 -07:00
Devin Bidwell
f60c9b32a8 minor version bump 2025-12-09 16:50:23 -07:00
Devin Bidwell
0cddb3e8c8 The Cows are all working. Moo. 2025-12-09 16:31:24 -07:00
Devin Bidwell
c3986ab4d9 VariableManager lifetime errors 2025-12-09 16:05:40 -07:00
Devin Bidwell
f54214acb9 Most of all the errors are gone 2025-12-09 13:59:54 -07:00
Devin Bidwell
d9a7a31306 Lifetimes are declared, now I gotta fix the lifetime issues 2025-12-09 13:51:54 -07:00
Devin Bidwell
d40b759442 TEST -- use Cow instead of String for tokens 2025-12-09 13:17:35 -07:00
Devin Bidwell
080b5320f7 Removed off-by-one calculations in the C# mod 2025-12-09 12:24:29 -07:00
Devin Bidwell
a50a45f0b4 More cleanup 2025-12-09 12:12:28 -07:00
Devin Bidwell
c531f673a5 Remove quickerror in favor of thiserror 2025-12-09 11:32:14 -07:00
Devin Bidwell
23c2ba4134 Fix visual bugs with new span logic 2025-12-09 02:21:56 -07:00
Devin Bidwell
7b7c1f7d29 Logos plugged into Parser 2025-12-09 02:15:43 -07:00
Devin Bidwell
72cf9ea042 wip 2025-12-09 01:43:12 -07:00
Devin Bidwell
fac36c756b Lexer impl done 2025-12-08 23:19:23 -07:00
Devin Bidwell
115a57128c Before error type refactor 2025-12-08 22:50:20 -07:00
Devin Bidwell
6afeec6da2 First pass getting a logos tokenizer up and running 2025-12-08 21:06:42 -07:00
Devin Bidwell
b6123219f8 Merge pull request #21 from dbidwell94/cysharp-removal 
Updates for Stationeers Beta branch
 2025-12-08 15:26:22 -07:00
Devin Bidwell
f38c15da9c Remove references to Unitask and Cysharp 2025-12-07 23:00:45 -07:00
Devin Bidwell
fdd1a311d5 Merge pull request #20 from dbidwell94/release 
Release
 2025-12-07 21:46:05 -07:00
Devin Bidwell
fb7ca0d4fd Update About.xml again 2025-12-07 21:45:07 -07:00
32 changed files with 2021 additions and 1848 deletions
Expand all files Collapse all files

34
Changelog.md Normal file
View File

@@ -0,0 +1,34 @@
# Changelog
[0.2.3]
- Fixed stack underflow with function invocations
- They are still "heavy", but they should work as expected now
- Fixed issue where syscall functions were not allowed as infix operators
[0.2.2]
- Fixed some formatting issues when converting Markdown to Text Mesh Pro for
Stationpedia
- Added support for ternary expressions
- `let i = someValue ? 4 : 5;`
- `i = someValue ? 4 : 5;`
- This greedily evaluates both sides, so side effects like calling functions
is not recommended i.e.
- `i = someValue : doSomething() : doSomethingElse();`
- Both sides will be evaluated before calling the `select` instruction
[0.2.1]
- Added support for `loadSlot` and `setSlot`
- Fixed bug where syscalls like `max(1, 2)` were not allowed in assignment expressions
[0.2.0]
- Completely re-wrote the tokenizer to use `logos`
- Changed AST and Token data structures to use `Cow` instead of `String`
- Updated error reporting to use `thiserror` instead of `quickerror`
[0.1.2]
- Removed references to `Unitask`

8
ModData/About/About.xml
View File

@@ -1,14 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<ModMetadata xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Name>StationeersSlang</Name>
<Name>Slang</Name>
<Author>JoeDiertay</Author>
<Version>0.1.1</Version>
<Version>0.2.3</Version>
<Description>
[h1]Slang: High-Level Programming for Stationeers[/h1]
Stop writing assembly. Start writing code.
Slang (Stationeers Language) brings modern programming to Stationeers. It allows you to write scripts using a familiar C-style syntax (variables, functions, if/else, loops) directly in the in-game editor. When you hit confirm, Slang compiles your code into optimized IC10 MIPS assembly instantly.
Slang (Stationeers Language) brings modern programming to Stationeers. It allows you to write scripts using a familiar C-style syntax (variables, functions, if/else, loops) directly in the in-game editor. When you hit confirm, Slang compiles your code into IC10 instantly.
[b]NOTE: This project is in BETA. Expect updates and changes![/b]
@@ -91,7 +91,7 @@ A: Yes! Slang does not modify any existing IC10 code, it is only a compiler. As
<OrderBefore WorkshopHandle="3592775931" />
<InGameDescription><![CDATA[
<size=30><color=#ffff00>Slang - High Level Language Compiler</color></size>
A modern programming experience for Stationeers. Write C-style code that compiles to MIPS assembly instantly.
A modern programming experience for Stationeers. Write C-style code that compiles to IC10 instantly.
<color=#ffa500><b>Features</b></color>
- <b>In-Game Compilation:</b> Write high-level logic directly in the chip editor.

13
csharp_mod/Extensions.cs
View File

@@ -55,7 +55,7 @@ public static unsafe class SlangExtensions
var color = GetColorForKind(token.token_kind);
int colIndex = token.column - 1;
int colIndex = token.column;
if (colIndex < 0)
colIndex = 0;
@@ -100,10 +100,10 @@ public static unsafe class SlangExtensions
Severity = item.severity,
Range = new Slang.Range
{
EndCol = Math.Max(item.range.end_col - 2, 0),
EndLine = item.range.end_line - 1,
StartCol = Math.Max(item.range.start_col - 2, 0),
StartLine = item.range.end_line - 1,
EndCol = Math.Max(item.range.end_col, 0),
EndLine = item.range.end_line,
StartCol = Math.Max(item.range.start_col, 0),
StartLine = item.range.start_line,
},
}
);
@@ -134,6 +134,9 @@ public static unsafe class SlangExtensions
case 7: // (punctuation)
return SlangFormatter.ColorDefault;
case 8: // Comments
return SlangFormatter.ColorComment;
case 10: // (syscalls)
return SlangFormatter.ColorFunction;

28
csharp_mod/Formatter.cs
View File

@@ -4,7 +4,7 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using Cysharp.Threading.Tasks;
using System.Threading.Tasks;
using StationeersIC10Editor;
public class SlangFormatter : ICodeFormatter
@@ -98,29 +98,32 @@ public class SlangFormatter : ICodeFormatter
inputSrc = this.RawText;
}
HandleLsp(inputSrc, token).Forget();
_ = HandleLsp(inputSrc, token);
}
private async UniTaskVoid HandleLsp(string inputSrc, CancellationToken cancellationToken)
private async Task HandleLsp(string inputSrc, CancellationToken cancellationToken)
{
try
{
await UniTask.SwitchToThreadPool();
if (cancellationToken.IsCancellationRequested)
return;
await Task.Delay(200, cancellationToken: cancellationToken);
if (cancellationToken.IsCancellationRequested)
return;
await System.Threading.Tasks.Task.Delay(200, cancellationToken: cancellationToken);
if (cancellationToken.IsCancellationRequested)
return;
var dict = Marshal
// Running this potentially CPU intensive work on a background thread.
var dict = await Task.Run(
() =>
{
return Marshal
.DiagnoseSource(inputSrc)
.GroupBy(d => d.Range.StartLine)
.ToDictionary(g => g.Key);
await UniTask.Yield(PlayerLoopTiming.Update, cancellationToken);
},
cancellationToken
);
ApplyDiagnostics(dict);
}
@@ -136,7 +139,6 @@ public class SlangFormatter : ICodeFormatter
{
HashSet<uint> linesToRefresh;
// CRITICAL FIX FOR LINE SHIFTS:
// If the line count has changed (lines added/deleted), indices have shifted.
// We must refresh ALL lines to ensure any line that shifted into a new position
// gets scrubbed of its old visual state.

7
csharp_mod/Patches.cs
View File

@@ -12,6 +12,7 @@ public static class SlangPatches
{
private static ProgrammableChipMotherboard? _currentlyEditingMotherboard;
private static AsciiString? _motherboardCachedCode;
private static Guid? _currentlyEditingGuid;
[HarmonyPatch(
typeof(ProgrammableChipMotherboard),
@@ -32,11 +33,13 @@ public static class SlangPatches
return;
}
var thisRef = Guid.NewGuid();
var thisRef = _currentlyEditingGuid ?? Guid.NewGuid();
// Ensure we cache this compiled code for later retreival.
GlobalCode.SetSource(thisRef, result);
_currentlyEditingGuid = null;
// Append REF to the bottom
compiled += $"\n{GlobalCode.SLANG_REF}{thisRef}";
result = compiled;
@@ -77,6 +80,7 @@ public static class SlangPatches
return;
}
_currentlyEditingGuid = sourceRef;
var slangSource = GlobalCode.GetSource(sourceRef);
if (string.IsNullOrEmpty(slangSource))
@@ -223,6 +227,7 @@ public static class SlangPatches
_currentlyEditingMotherboard = null;
_motherboardCachedCode = null;
_currentlyEditingGuid = null;
}
[HarmonyPatch(typeof(Stationpedia), nameof(Stationpedia.Regenerate))]

14
csharp_mod/TmpFormatter.cs
View File

@@ -26,12 +26,18 @@ public static class TextMeshProFormatter
RegexOptions.Singleline
);
// 3. Handle Headers (## Header)
// Convert ## Header to large bold text
text = Regex.Replace(
text,
@"^##(\s+)?(.+)$",
"<size=120%><b>$1</b></size>",
@"^\s*##\s+(.+)$",
"<size=110%><color=#ffffff><b>$1</b></color></size>",
RegexOptions.Multiline
);
// 3. Handle # Headers SECOND (General)
text = Regex.Replace(
text,
@"^\s*#\s+(.+)$",
"<size=120%><color=#ffffff><b>$1</b></color></size>",
RegexOptions.Multiline
);

6
csharp_mod/stationeersSlang.csproj
View File

@@ -5,7 +5,7 @@
<Nullable>enable</Nullable>
<AssemblyName>StationeersSlang</AssemblyName>
<Description>Slang Compiler Bridge</Description>
<Version>0.1.1</Version>
<Version>0.2.3</Version>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<LangVersion>latest</LangVersion>
</PropertyGroup>
@@ -39,10 +39,6 @@
<HintPath>./ref/Assembly-CSharp.dll</HintPath>
<Private>False</Private>
</Reference>
<Reference Include="UniTask">
<HintPath>./ref/UniTask.dll</HintPath>
<Private>False</Private>
</Reference>
<Reference Include="IC10Editor.dll">
<HintPath>./ref/IC10Editor.dll</HintPath>

109
rust_compiler/Cargo.lock generated
View File

@@ -28,6 +28,15 @@ dependencies = [
"version_check",
]
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "anstream"
version = "0.6.21"
@@ -114,6 +123,12 @@ dependencies = [
"windows-link",
]
[[package]]
name = "beef"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a8241f3ebb85c056b509d4327ad0358fbbba6ffb340bf388f26350aeda225b1"
[[package]]
name = "bitflags"
version = "1.3.2"
@@ -257,8 +272,8 @@ dependencies = [
"lsp-types",
"parser",
"pretty_assertions",
"quick-error",
"rust_decimal",
"thiserror",
"tokenizer",
]
@@ -327,6 +342,12 @@ dependencies = [
"bitflags",
]
[[package]]
name = "fnv"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "funty"
version = "2.0.0"
@@ -434,6 +455,40 @@ version = "0.2.178"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37c93d8daa9d8a012fd8ab92f088405fb202ea0b6ab73ee2482ae66af4f42091"
[[package]]
name = "logos"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a790d11254054e5dc83902dba85d253ff06ceb0cfafb12be8773435cb9dfb4f4"
dependencies = [
"logos-derive",
]
[[package]]
name = "logos-codegen"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f60337c43a38313b58871f8d5d76872b8e17aa9d51fad494b5e76092c0ce05f5"
dependencies = [
"beef",
"fnv",
"proc-macro2",
"quote",
"regex-automata",
"regex-syntax",
"rustc_version",
"syn 2.0.111",
]
[[package]]
name = "logos-derive"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d151b2ae667f69e10b8738f5cac0c746faa22b2e15ea7e83b55476afec3767dc"
dependencies = [
"logos-codegen",
]
[[package]]
name = "lsp-types"
version = "0.97.0"
@@ -516,7 +571,7 @@ dependencies = [
"helpers",
"lsp-types",
"pretty_assertions",
"quick-error",
"thiserror",
"tokenizer",
]
@@ -593,12 +648,6 @@ dependencies = [
"syn 1.0.109",
]
[[package]]
name = "quick-error"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
[[package]]
name = "quote"
version = "1.0.42"
@@ -644,6 +693,23 @@ dependencies = [
"getrandom",
]
[[package]]
name = "regex-automata"
version = "0.4.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58"
[[package]]
name = "rend"
version = "0.4.2"
@@ -843,7 +909,7 @@ checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
[[package]]
name = "slang"
version = "0.1.1"
version = "0.2.3"
dependencies = [
"anyhow",
"clap",
@@ -851,9 +917,9 @@ dependencies = [
"helpers",
"lsp-types",
"parser",
"quick-error",
"rust_decimal",
"safer-ffi",
"thiserror",
"tokenizer",
]
@@ -926,6 +992,26 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "thiserror"
version = "2.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f63587ca0f12b72a0600bcba1d40081f830876000bb46dd2337a3051618f4fc8"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "2.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ff15c8ecd7de3849db632e14d18d2571fa09dfc5ed93479bc4485c7a517c913"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.111",
]
[[package]]
name = "tinyvec"
version = "1.10.0"
@@ -947,9 +1033,10 @@ version = "0.1.0"
dependencies = [
"anyhow",
"helpers",
"logos",
"lsp-types",
"quick-error",
"rust_decimal",
"thiserror",
]
[[package]]

8
rust_compiler/Cargo.toml
View File

@@ -1,13 +1,13 @@
[package]
name = "slang"
version = "0.1.1"
version = "0.2.3"
edition = "2021"
[workspace]
members = ["libs/*"]
[workspace.dependencies]
quick-error = "2"
thiserror = "2"
rust_decimal = "1"
safer-ffi = { version = "0.1" } # Safely share structs in memory between C# and Rust
lsp-types = { version = "0.97" } # Allows for LSP style reporting to the frontend
@@ -36,13 +36,13 @@ crate-type = ["cdylib", "rlib"]
[dependencies]
clap = { version = "^4.5", features = ["derive"] }
lsp-types = { workspace = true }
quick-error = { workspace = true }
thiserror = { workspace = true }
rust_decimal = { workspace = true }
tokenizer = { path = "libs/tokenizer" }
parser = { path = "libs/parser" }
compiler = { path = "libs/compiler" }
helpers = { path = "libs/helpers" }
safer-ffi = { workspace = true }
anyhow = { version = "^1.0", features = ["backtrace"] }
[dev-dependencies]
anyhow = { version = "^1.0", features = ["backtrace"] }

2
rust_compiler/libs/compiler/Cargo.toml
View File

@@ -4,7 +4,7 @@ version = "0.1.0"
edition = "2024"
[dependencies]
quick-error = { workspace = true }
thiserror = { workspace = true }
parser = { path = "../parser" }
tokenizer = { path = "../tokenizer" }
helpers = { path = "../helpers" }

63
rust_compiler/libs/compiler/src/test/binary_expression.rs
View File

@@ -1,9 +1,10 @@
use crate::compile;
use anyhow::Result;
use indoc::indoc;
use pretty_assertions::assert_eq;
#[test]
fn simple_binary_expression() -> anyhow::Result<()> {
fn simple_binary_expression() -> Result<()> {
let compiled = compile! {
debug
"
@@ -26,7 +27,7 @@ fn simple_binary_expression() -> anyhow::Result<()> {
}
#[test]
fn nested_binary_expressions() -> anyhow::Result<()> {
fn nested_binary_expressions() -> Result<()> {
let compiled = compile! {
debug
"
@@ -51,6 +52,8 @@ fn nested_binary_expressions() -> anyhow::Result<()> {
add r1 r10 r9
mul r2 r1 r8
move r15 r2
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
@@ -71,7 +74,7 @@ fn nested_binary_expressions() -> anyhow::Result<()> {
}
#[test]
fn stress_test_constant_folding() -> anyhow::Result<()> {
fn stress_test_constant_folding() -> Result<()> {
let compiled = compile! {
debug
"
@@ -94,7 +97,7 @@ fn stress_test_constant_folding() -> anyhow::Result<()> {
}
#[test]
fn test_constant_folding_with_variables_mixed_in() -> anyhow::Result<()> {
fn test_constant_folding_with_variables_mixed_in() -> Result<()> {
let compiled = compile! {
debug
r#"
@@ -120,3 +123,55 @@ fn test_constant_folding_with_variables_mixed_in() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn test_ternary_expression() -> Result<()> {
let compiled = compile! {
debug
r#"
let i = 1 > 2 ? 15 : 20;
"#
};
assert_eq!(
compiled,
indoc! {
"
j main
main:
sgt r1 1 2
select r2 r1 15 20
move r8 r2 #i
"
}
);
Ok(())
}
#[test]
fn test_ternary_expression_assignment() -> Result<()> {
let compiled = compile! {
debug
r#"
let i = 0;
i = 1 > 2 ? 15 : 20;
"#
};
assert_eq!(
compiled,
indoc! {
"
j main
main:
move r8 0 #i
sgt r1 1 2
select r2 r1 15 20
move r8 r2 #i
"
}
);
Ok(())
}

39
rust_compiler/libs/compiler/src/test/declaration_function_invocation.rs
View File

@@ -17,6 +17,7 @@ fn no_arguments() -> anyhow::Result<()> {
j main
doSomething:
push ra
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
@@ -34,14 +35,17 @@ fn no_arguments() -> anyhow::Result<()> {
#[test]
fn let_var_args() -> anyhow::Result<()> {
// !IMPORTANT this needs to be stabilized as it currently incorrectly calculates sp offset at
// both ends of the cleanup lifecycle
let compiled = compile! {
debug
"
fn doSomething(arg1) {};
fn mul2(arg1) {
return arg1 * 2;
};
loop {
let arg1 = 123;
let i = doSomething(arg1);
let i = mul2(arg1);
i = i ** 2;
}
"
};
@@ -50,23 +54,31 @@ fn let_var_args() -> anyhow::Result<()> {
indoc! {
"
j main
doSomething:
mul2:
pop r8 #arg1
push ra
mul r1 r8 2
move r15 r1
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
j ra
main:
L2:
move r8 123 #arg1
push r8
push r8
jal doSomething
jal mul2
sub r0 sp 1
get r8 db r0
sub sp sp 1
move r9 r15 #i
sub sp sp 1
pow r1 r9 2
move r9 r1 #i
j L2
L3:
"
}
);
@@ -97,7 +109,9 @@ fn inline_literal_args() -> anyhow::Result<()> {
let compiled = compile! {
debug
"
fn doSomething(arg1, arg2) {};
fn doSomething(arg1, arg2) {
return 5;
};
let thisVariableShouldStayInPlace = 123;
let returnedValue = doSomething(12, 34);
"
@@ -112,6 +126,9 @@ fn inline_literal_args() -> anyhow::Result<()> {
pop r8 #arg2
pop r9 #arg1
push ra
move r15 5 #returnValue
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
@@ -126,7 +143,6 @@ fn inline_literal_args() -> anyhow::Result<()> {
get r8 db r0
sub sp sp 1
move r9 r15 #returnedValue
sub sp sp 1
"
}
);
@@ -154,6 +170,7 @@ fn mixed_args() -> anyhow::Result<()> {
pop r8 #arg2
pop r9 #arg1
push ra
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
@@ -168,7 +185,6 @@ fn mixed_args() -> anyhow::Result<()> {
get r8 db r0
sub sp sp 1
move r9 r15 #returnValue
sub sp sp 1
"
}
);
@@ -198,6 +214,8 @@ fn with_return_statement() -> anyhow::Result<()> {
pop r8 #arg1
push ra
move r15 456 #returnValue
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
@@ -233,6 +251,7 @@ fn with_negative_return_literal() -> anyhow::Result<()> {
doSomething:
push ra
move r15 -1 #returnValue
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1

2
rust_compiler/libs/compiler/src/test/declaration_literal.rs
View File

@@ -133,6 +133,8 @@ fn test_boolean_return() -> anyhow::Result<()> {
getTrue:
push ra
move r15 1 #returnValue
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1

44
rust_compiler/libs/compiler/src/test/function_declaration.rs
View File

@@ -21,6 +21,7 @@ fn test_function_declaration_with_spillover_params() -> anyhow::Result<()> {
pop r13 #arg4
pop r14 #arg3
push ra
L1:
sub r0 sp 1
get ra db r0
sub sp sp 3
@@ -31,6 +32,48 @@ fn test_function_declaration_with_spillover_params() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn test_early_return() -> anyhow::Result<()> {
let compiled = compile!(debug r#"
// This is a test function declaration with no body
fn doSomething() {
if (1 == 1) {
return;
}
let i = 1 + 2;
return;
};
doSomething();
"#);
assert_eq!(
compiled,
indoc! {
"
j main
doSomething:
push ra
seq r1 1 1
beq r1 0 L2
j L1
L2:
move r8 3 #i
j L1
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1
j ra
main:
jal doSomething
move r1 r15 #__binary_temp_2
"
}
);
Ok(())
}
#[test]
fn test_function_declaration_with_register_params() -> anyhow::Result<()> {
let compiled = compile!(debug r#"
@@ -47,6 +90,7 @@ fn test_function_declaration_with_register_params() -> anyhow::Result<()> {
pop r8 #arg2
pop r9 #arg1
push ra
L1:
sub r0 sp 1
get ra db r0
sub sp sp 1

26
rust_compiler/libs/compiler/src/test/math_syscall.rs
View File

@@ -243,6 +243,32 @@ fn test_max() -> Result<()> {
Ok(())
}
#[test]
fn test_max_from_game() -> Result<()> {
let compiled = compile! {
debug
r#"
let item = 0;
item = max(1 + 2, 2);
"#
};
assert_eq!(
compiled,
indoc! {
"
j main
main:
move r8 0 #item
max r15 3 2
move r8 r15 #item
"
}
);
Ok(())
}
#[test]
fn test_min() -> Result<()> {
let compiled = compile! {

4
rust_compiler/libs/compiler/src/test/mod.rs
View File

@@ -22,7 +22,7 @@ macro_rules! compile {
(result $source:expr) => {{
let mut writer = std::io::BufWriter::new(Vec::new());
let compiler = crate::Compiler::new(
parser::Parser::new(tokenizer::Tokenizer::from(String::from($source))),
parser::Parser::new(tokenizer::Tokenizer::from($source)),
&mut writer,
Some(crate::CompilerConfig { debug: true }),
);
@@ -32,7 +32,7 @@ macro_rules! compile {
(debug $source:expr) => {{
let mut writer = std::io::BufWriter::new(Vec::new());
let compiler = crate::Compiler::new(
parser::Parser::new(tokenizer::Tokenizer::from(String::from($source))),
parser::Parser::new(tokenizer::Tokenizer::from($source)),
&mut writer,
Some(crate::CompilerConfig { debug: true }),
);

51
rust_compiler/libs/compiler/src/test/syscall.rs
View File

@@ -157,3 +157,54 @@ fn test_load_from_device() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn test_load_from_slot() -> anyhow::Result<()> {
let compiled = compile! {
debug
r#"
device airCon = "d0";
let setting = ls(airCon, 0, "Occupied");
"#
};
assert_eq!(
compiled,
indoc! {
"
j main
main:
ls r15 d0 0 Occupied
move r8 r15 #setting
"
}
);
Ok(())
}
#[test]
fn test_set_slot() -> anyhow::Result<()> {
let compiled = compile! {
debug
r#"
device airCon = "d0";
ss(airCon, 0, "Occupied", true);
"#
};
assert_eq!(
compiled,
indoc! {
"
j main
main:
ss d0 0 Occupied 1
"
}
);
Ok(())
}

676
rust_compiler/libs/compiler/src/v1.rs
View File

File diff suppressed because it is too large Load Diff

113
rust_compiler/libs/compiler/src/variable_manager.rs
View File

@@ -5,28 +5,28 @@
use lsp_types::{Diagnostic, DiagnosticSeverity};
use parser::tree_node::{Literal, Span};
use quick_error::quick_error;
use std::collections::{HashMap, VecDeque};
use std::{
borrow::Cow,
collections::{HashMap, VecDeque},
};
use thiserror::Error;
const TEMP: [u8; 7] = [1, 2, 3, 4, 5, 6, 7];
const PERSIST: [u8; 7] = [8, 9, 10, 11, 12, 13, 14];
quick_error! {
#[derive(Debug)]
pub enum Error {
DuplicateVariable(var: String, span: Option<Span>) {
display("{var} already exists.")
}
UnknownVariable(var: String, span: Option<Span>) {
display("{var} does not exist.")
}
Unknown(reason: String, span: Option<Span>) {
display("{reason}")
}
}
#[derive(Error, Debug)]
pub enum Error<'a> {
#[error("{0} already exists.")]
DuplicateVariable(Cow<'a, str>, Option<Span>),
#[error("{0} does not exist.")]
UnknownVariable(Cow<'a, str>, Option<Span>),
#[error("{0}")]
Unknown(Cow<'a, str>, Option<Span>),
}
impl From<Error> for lsp_types::Diagnostic {
impl<'a> From<Error<'a>> for lsp_types::Diagnostic {
fn from(value: Error) -> Self {
match value {
Error::DuplicateVariable(_, span)
@@ -52,8 +52,8 @@ pub enum LocationRequest {
Stack,
}
#[derive(Clone)]
pub enum VariableLocation {
#[derive(Clone, Debug)]
pub enum VariableLocation<'a> {
/// Represents a temporary register (r1 - r7)
Temporary(u8),
/// Represents a persistant register (r8 - r14)
@@ -61,20 +61,20 @@ pub enum VariableLocation {
/// Represents a a stack offset (current stack - offset = variable loc)
Stack(u16),
/// Represents a constant value and should be directly substituted as such.
Constant(Literal),
Constant(Literal<'a>),
/// Represents a device pin. This will contain the exact `d0-d5` string
Device(String),
Device(Cow<'a, str>),
}
pub struct VariableScope<'a> {
pub struct VariableScope<'a, 'b> {
temporary_vars: VecDeque<u8>,
persistant_vars: VecDeque<u8>,
var_lookup_table: HashMap<String, VariableLocation>,
var_lookup_table: HashMap<Cow<'a, str>, VariableLocation<'a>>,
stack_offset: u16,
parent: Option<&'a VariableScope<'a>>,
parent: Option<&'b VariableScope<'a, 'b>>,
}
impl<'a> Default for VariableScope<'a> {
impl<'a, 'b> Default for VariableScope<'a, 'b> {
fn default() -> Self {
Self {
parent: None,
@@ -86,7 +86,7 @@ impl<'a> Default for VariableScope<'a> {
}
}
impl<'a> VariableScope<'a> {
impl<'a, 'b> VariableScope<'a, 'b> {
#[allow(dead_code)]
pub const TEMP_REGISTER_COUNT: u8 = 7;
pub const PERSIST_REGISTER_COUNT: u8 = 7;
@@ -94,22 +94,24 @@ impl<'a> VariableScope<'a> {
pub const RETURN_REGISTER: u8 = 15;
pub const TEMP_STACK_REGISTER: u8 = 0;
pub fn registers(&self) -> impl Iterator<Item = &u8> {
self.var_lookup_table
.values()
.filter(|val| {
matches!(
val,
VariableLocation::Temporary(_) | VariableLocation::Persistant(_)
)
})
.map(|loc| match loc {
VariableLocation::Persistant(reg) | VariableLocation::Temporary(reg) => reg,
_ => unreachable!(),
})
pub fn registers(&self) -> Vec<u8> {
let mut used = Vec::new();
for r in TEMP {
if !self.temporary_vars.contains(&r) {
used.push(r);
}
}
pub fn scoped(parent: &'a VariableScope<'a>) -> Self {
for r in PERSIST {
if !self.persistant_vars.contains(&r) {
used.push(r);
}
}
used
}
pub fn scoped(parent: &'b VariableScope<'a, 'b>) -> Self {
Self {
parent: Option::Some(parent),
temporary_vars: parent.temporary_vars.clone(),
@@ -126,12 +128,11 @@ impl<'a> VariableScope<'a> {
/// to the stack.
pub fn add_variable(
&mut self,
var_name: impl Into<String>,
var_name: Cow<'a, str>,
location: LocationRequest,
span: Option<Span>,
) -> Result<VariableLocation, Error> {
let var_name = var_name.into();
if self.var_lookup_table.contains_key(var_name.as_str()) {
) -> Result<VariableLocation<'a>, Error<'a>> {
if self.var_lookup_table.contains_key(&var_name) {
return Err(Error::DuplicateVariable(var_name, span));
}
let var_location = match location {
@@ -166,11 +167,10 @@ impl<'a> VariableScope<'a> {
pub fn define_const(
&mut self,
var_name: impl Into<String>,
value: Literal,
var_name: Cow<'a, str>,
value: Literal<'a>,
span: Option<Span>,
) -> Result<VariableLocation, Error> {
let var_name = var_name.into();
) -> Result<VariableLocation<'a>, Error<'a>> {
if self.var_lookup_table.contains_key(&var_name) {
return Err(Error::DuplicateVariable(var_name, span));
}
@@ -183,13 +183,11 @@ impl<'a> VariableScope<'a> {
pub fn get_location_of(
&self,
var_name: impl Into<String>,
var_name: &Cow<'a, str>,
span: Option<Span>,
) -> Result<VariableLocation, Error> {
let var_name = var_name.into();
) -> Result<VariableLocation<'a>, Error<'a>> {
// 1. Check this scope
if let Some(var) = self.var_lookup_table.get(var_name.as_str()) {
if let Some(var) = self.var_lookup_table.get(var_name) {
if let VariableLocation::Stack(inserted_at_offset) = var {
// Return offset relative to CURRENT sp
return Ok(VariableLocation::Stack(
@@ -210,7 +208,7 @@ impl<'a> VariableScope<'a> {
return Ok(loc);
}
Err(Error::UnknownVariable(var_name, span))
Err(Error::UnknownVariable(var_name.clone(), span))
}
pub fn has_parent(&self) -> bool {
@@ -220,11 +218,10 @@ impl<'a> VariableScope<'a> {
#[allow(dead_code)]
pub fn free_temp(
&mut self,
var_name: impl Into<String>,
var_name: Cow<'a, str>,
span: Option<Span>,
) -> Result<(), Error> {
let var_name = var_name.into();
let Some(location) = self.var_lookup_table.remove(var_name.as_str()) else {
) -> Result<(), Error<'a>> {
let Some(location) = self.var_lookup_table.remove(&var_name) else {
return Err(Error::UnknownVariable(var_name, span));
};
@@ -234,7 +231,7 @@ impl<'a> VariableScope<'a> {
}
VariableLocation::Persistant(_) => {
return Err(Error::UnknownVariable(
String::from("Attempted to free a `let` variable."),
Cow::from("Attempted to free a `let` variable."),
span,
));
}

41
rust_compiler/libs/compiler/test_files/script.slang Normal file
View File

@@ -0,0 +1,41 @@
// Pressure numbers are in KPa
device self = "db";
device emergencyRelief = "d0";
device greenhouseSensor = "d1";
device recycleValve = "d2";
const MAX_INTERIOR_PRESSURE = 80;
const MAX_INTERIOR_TEMP = 28c;
const MIN_INTERIOR_PRESSURE = 75;
const MIN_INTERIOR_TEMP = 25c;
const daylightSensor = 1076425094;
const growLight = hash("StructureGrowLight");
const wallLight = hash("StructureLightLong");
const lightRound = hash("StructureLightRound");
let shouldPurge = false;
loop {
let interiorPress = greenhouseSensor.Pressure;
let interiorTemp = greenhouseSensor.Temperature;
shouldPurge = (
interiorPress > MAX_INTERIOR_PRESSURE ||
interiorTemp > MAX_INTERIOR_TEMP
) || shouldPurge;
emergencyRelief.On = shouldPurge;
recycleValve.On = !shouldPurge;
if (shouldPurge && (interiorPress < MIN_INTERIOR_PRESSURE && interiorTemp < MIN_INTERIOR_TEMP)) {
shouldPurge = false;
}
let solarAngle = lb(daylightSensor, "SolarAngle", "Average");
let isDaylight = solarAngle < 90;
sb(growLight, "On", isDaylight);
sb(wallLight, "On", !isDaylight);
sb(lightRound, "On", !isDaylight);
}

62
rust_compiler/libs/helpers/src/macros.rs
View File

@@ -3,15 +3,10 @@ macro_rules! documented {
// -------------------------------------------------------------------------
// Internal Helper: Filter doc comments
// -------------------------------------------------------------------------
// Case 1: Doc comment. Return Some("string").
// We match the specific structure of a doc attribute.
(@doc_filter #[doc = $doc:expr]) => {
Some($doc)
};
// Case 2: Other attributes (derives, etc.). Return None.
// We catch any other token sequence inside the brackets.
(@doc_filter #[$($attr:tt)*]) => {
None
};
@@ -30,23 +25,59 @@ macro_rules! documented {
};
// -------------------------------------------------------------------------
// Main Macro Entry Point
// Entry Point 1: Enum with a single Lifetime (e.g. enum Foo<'a>)
// -------------------------------------------------------------------------
(
$(#[$enum_attr:meta])* $vis:vis enum $name:ident < $lt:lifetime > {
$($body:tt)*
}
) => {
documented!(@generate
meta: [$(#[$enum_attr])*],
vis: [$vis],
name: [$name],
generics: [<$lt>],
body: [$($body)*]
);
};
// -------------------------------------------------------------------------
// Entry Point 2: Regular Enum (No Generics)
// -------------------------------------------------------------------------
(
$(#[$enum_attr:meta])* $vis:vis enum $name:ident {
$($body:tt)*
}
) => {
documented!(@generate
meta: [$(#[$enum_attr])*],
vis: [$vis],
name: [$name],
generics: [],
body: [$($body)*]
);
};
// -------------------------------------------------------------------------
// Code Generator (Shared Logic)
// -------------------------------------------------------------------------
(@generate
meta: [$(#[$enum_attr:meta])*],
vis: [$vis:vis],
name: [$name:ident],
generics: [$($generics:tt)*],
body: [
$(
// Capture attributes as a sequence of token trees inside brackets
// to avoid "local ambiguity" and handle multi-token attributes (like doc="...").
$(#[ $($variant_attr:tt)* ])*
$variant:ident
$( ($($tuple:tt)*) )?
$( {$($structure:tt)*} )?
),* $(,)?
}
]
) => {
// 1. Generate the actual Enum definition
// 1. Generate the Enum Definition
$(#[$enum_attr])*
$vis enum $name {
$vis enum $name $($generics)* {
$(
$(#[ $($variant_attr)* ])*
$variant
@@ -55,20 +86,19 @@ macro_rules! documented {
)*
}
// 2. Implement the Documentation Trait
impl Documentation for $name {
// 2. Implement Documentation Trait
// We apply the captured generics (e.g., <'a>) to both the impl and the type
impl $($generics)* Documentation for $name $($generics)* {
fn docs(&self) -> String {
match self {
$(
documented!(@arm $name $variant $( ($($tuple)*) )? $( {$($structure)*} )? ) => {
// Create a temporary array of Option<&str> for all attributes
let doc_lines: &[Option<&str>] = &[
$(
documented!(@doc_filter #[ $($variant_attr)* ])
),*
];
// Filter out the Nones (non-doc attributes), join, and return
doc_lines.iter()
.filter_map(|&d| d)
.collect::<Vec<_>>()
@@ -80,7 +110,6 @@ macro_rules! documented {
}
}
// 3. Implement Static Documentation Provider
#[allow(dead_code)]
fn get_all_documentation() -> Vec<(&'static str, String)> {
vec![
@@ -88,7 +117,6 @@ macro_rules! documented {
(
stringify!($variant),
{
// Re-use the same extraction logic
let doc_lines: &[Option<&str>] = &[
$(
documented!(@doc_filter #[ $($variant_attr)* ])

6
rust_compiler/libs/helpers/src/syscall.rs
View File

@@ -9,9 +9,11 @@ macro_rules! with_syscalls {
"load",
"loadBatched",
"loadBatchedNamed",
"loadSlot",
"set",
"setBatched",
"setBatchedNamed",
"setSlot",
"acos",
"asin",
"atan",
@@ -32,9 +34,11 @@ macro_rules! with_syscalls {
"l",
"lb",
"lbn",
"ls",
"s",
"sb",
"sbn"
"sbn",
"ss"
);
};
}

2
rust_compiler/libs/parser/Cargo.toml
View File

@@ -4,10 +4,10 @@ version = "0.1.0"
edition = "2024"
[dependencies]
quick-error = { workspace = true }
tokenizer = { path = "../tokenizer" }
helpers = { path = "../helpers" }
lsp-types = { workspace = true }
thiserror = { workspace = true }
[dev-dependencies]

554
rust_compiler/libs/parser/src/lib.rs
View File

File diff suppressed because it is too large Load Diff

104
rust_compiler/libs/parser/src/sys_call.rs
View File

@@ -4,73 +4,73 @@ use helpers::prelude::*;
documented! {
#[derive(Debug, PartialEq, Eq)]
pub enum Math {
pub enum Math<'a> {
/// Returns the angle in radians whose cosine is the specified number.
/// ## IC10
/// `acos r? a(r?|num)`
/// ## Slang
/// `let item = acos(number|var|expression);`
Acos(Box<Spanned<Expression>>),
Acos(Box<Spanned<Expression<'a>>>),
/// Returns the angle in radians whose sine is the specified number.
/// ## IC10
/// `asin r? a(r?|num)`
/// ## Slang
/// `let item = asin(number|var|expression);`
Asin(Box<Spanned<Expression>>),
Asin(Box<Spanned<Expression<'a>>>),
/// Returns the angle in radians whose tangent is the specified number.
/// ## IC10
/// `atan r? a(r?|num)`
/// ## Slang
/// `let item = atan(number|var|expression);`
Atan(Box<Spanned<Expression>>),
Atan(Box<Spanned<Expression<'a>>>),
/// Returns the angle in radians whose tangent is the quotient of the specified numbers.
/// ## IC10
/// `atan2 r? a(r?|num) b(r?|num)`
/// ## Slang
/// `let item = atan2((number|var|expression), (number|var|expression));`
Atan2(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Atan2(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
/// Gets the absolute value of a number.
/// ## IC10
/// `abs r? a(r?|num)`
/// ## Slang
/// `let item = abs((number|var|expression));`
Abs(Box<Spanned<Expression>>),
Abs(Box<Spanned<Expression<'a>>>),
/// Rounds a number up to the nearest whole number.
/// ## IC10
/// `ceil r? a(r?|num)`
/// ## Slang
/// `let item = ceil((number|var|expression));`
Ceil(Box<Spanned<Expression>>),
Ceil(Box<Spanned<Expression<'a>>>),
/// Returns the cosine of the specified angle in radians.
/// ## IC10
/// `cos r? a(r?|num)`
/// ## Slang
/// `let item = cos((number|var|expression));`
Cos(Box<Spanned<Expression>>),
Cos(Box<Spanned<Expression<'a>>>),
/// Rounds a number down to the nearest whole number.
/// ## IC10
/// `floor r? a(r?|num)`
/// ## Slang
/// `let item = floor((number|var|expression));`
Floor(Box<Spanned<Expression>>),
Floor(Box<Spanned<Expression<'a>>>),
/// Computes the natural logarithm of a number.
/// ## IC10
/// `log r? a(r?|num)`
/// ## Slang
/// `let item = log((number|var|expression));`
Log(Box<Spanned<Expression>>),
Log(Box<Spanned<Expression<'a>>>),
/// Computes the maximum of two numbers.
/// ## IC10
/// `max r? a(r?|num) b(r?|num)`
/// ## Slang
/// `let item = max((number|var|expression), (number|var|expression));`
Max(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Max(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
/// Computes the minimum of two numbers.
/// ## IC10
/// `min r? a(r?|num) b(r?|num)`
/// ## Slang
/// `let item = min((number|var|expression), (number|var|expression));`
Min(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Min(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
/// Gets a random number between 0 and 1.
/// ## IC10
/// `rand r?`
@@ -82,29 +82,29 @@ documented! {
/// `sin r? a(r?|num)`
/// ## Slang
/// `let item = sin((number|var|expression));`
Sin(Box<Spanned<Expression>>),
Sin(Box<Spanned<Expression<'a>>>),
/// Computes the square root of a number.
/// ## IC10
/// `sqrt r? a(r?|num)`
/// ## Slang
/// `let item = sqrt((number|var|expression));`
Sqrt(Box<Spanned<Expression>>),
Sqrt(Box<Spanned<Expression<'a>>>),
/// Returns the tangent of the specified angle in radians.
/// ## IC10
/// `tan r? a(r?|num)`
/// ## Slang
/// `let item = tan((number|var|expression));`
Tan(Box<Spanned<Expression>>),
Tan(Box<Spanned<Expression<'a>>>),
/// Truncates a number by removing the decimal portion.
/// ## IC10
/// `trunc r? a(r?|num)`
/// ## Slang
/// `let item = trunc((number|var|expression));`
Trunc(Box<Spanned<Expression>>),
Trunc(Box<Spanned<Expression<'a>>>),
}
}
impl std::fmt::Display for Math {
impl<'a> std::fmt::Display for Math<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Math::Acos(a) => write!(f, "acos({})", a),
@@ -129,7 +129,7 @@ impl std::fmt::Display for Math {
documented! {
#[derive(Debug, PartialEq, Eq)]
pub enum System {
pub enum System<'a> {
/// Pauses execution for exactly 1 tick and then resumes.
/// ## IC10
/// `yield`
@@ -141,7 +141,7 @@ documented! {
/// `sleep a(r?|num)`
/// ## Slang
/// `sleep(number|var);`
Sleep(Box<Spanned<Expression>>),
Sleep(Box<Spanned<Expression<'a>>>),
/// Gets the in-game hash for a specific prefab name. NOTE! This call is COMPLETELY
/// optimized away unless you bind it to a `let` variable. If you use a `const` variable
/// however, the hash is correctly computed at compile time and substitued automatically.
@@ -155,7 +155,7 @@ documented! {
/// const compDoor = hash("StructureCompositeDoor");
/// setOnDeviceBatched(compDoor, "Lock", true);
/// ```
Hash(Spanned<Literal>),
Hash(Spanned<Literal<'a>>),
/// Represents a function which loads a device variable into a register.
/// ## IC10
/// `l r? d? var`
@@ -163,7 +163,7 @@ documented! {
/// `let item = load(deviceHash, "LogicType");`
/// `let item = l(deviceHash, "LogicType");`
/// `let item = deviceAlias.LogicType;`
LoadFromDevice(Spanned<LiteralOrVariable>, Spanned<Literal>),
LoadFromDevice(Spanned<LiteralOrVariable<'a>>, Spanned<Literal<'a>>),
/// Function which gets a LogicType from all connected network devices that match
/// the provided device hash and name, aggregating them via a batchMode
/// ## IC10
@@ -172,10 +172,10 @@ documented! {
/// `loadBatchedNamed(deviceHash, deviceName, "LogicType", "BatchMode");`
/// `lbn(deviceHash, deviceName, "LogicType", "BatchMode");`
LoadBatchNamed(
Spanned<LiteralOrVariable>,
Spanned<LiteralOrVariable>,
Spanned<Literal>,
Spanned<Literal>,
Spanned<LiteralOrVariable<'a>>,
Spanned<LiteralOrVariable<'a>>,
Spanned<Literal<'a>>,
Spanned<Literal<'a>>,
),
/// Loads a LogicType from all connected network devices, aggregating them via a
/// BatchMode
@@ -184,7 +184,7 @@ documented! {
/// ## Slang
/// `loadBatched(deviceHash, "Variable", "LogicType");`
/// `lb(deviceHash, "Variable", "LogicType");`
LoadBatch(Spanned<LiteralOrVariable>, Spanned<Literal>, Spanned<Literal>),
LoadBatch(Spanned<LiteralOrVariable<'a>>, Spanned<Literal<'a>>, Spanned<Literal<'a>>),
/// Represents a function which stores a setting into a specific device.
/// ## IC10
/// `s d? logicType r?`
@@ -192,7 +192,7 @@ documented! {
/// `set(deviceHash, "LogicType", (number|var));`
/// `s(deviceHash, "LogicType", (number|var));`
/// `deviceAlias.LogicType = (number|var);`
SetOnDevice(Spanned<LiteralOrVariable>, Spanned<Literal>, Box<Spanned<Expression>>),
SetOnDevice(Spanned<LiteralOrVariable<'a>>, Spanned<Literal<'a>>, Box<Spanned<Expression<'a>>>),
/// Represents a function which stores a setting to all devices that match
/// the given deviceHash
/// ## IC10
@@ -200,7 +200,7 @@ documented! {
/// ## Slang
/// `setBatched(deviceHash, "LogicType", (number|var));`
/// `sb(deviceHash, "LogicType", (number|var));`
SetOnDeviceBatched(Spanned<LiteralOrVariable>, Spanned<Literal>, Box<Spanned<Expression>>),
SetOnDeviceBatched(Spanned<LiteralOrVariable<'a>>, Spanned<Literal<'a>>, Box<Spanned<Expression<'a>>>),
/// Represents a function which stores a setting to all devices that match
/// both the given deviceHash AND the given nameHash
/// ## IC10
@@ -209,15 +209,39 @@ documented! {
/// `setBatchedNamed(deviceHash, nameHash, "LogicType", (number|var));`
/// `sbn(deviceHash, nameHash, "LogicType", (number|var));`
SetOnDeviceBatchedNamed(
Spanned<LiteralOrVariable>,
Spanned<LiteralOrVariable>,
Spanned<Literal>,
Box<Spanned<Expression>>,
Spanned<LiteralOrVariable<'a>>,
Spanned<LiteralOrVariable<'a>>,
Spanned<Literal<'a>>,
Box<Spanned<Expression<'a>>>,
),
/// Loads slot LogicSlotType from device into a variable
///
/// ## IC10
/// `ls r0 d0 2 Occupied`
/// ## Slang
/// `let isOccupied = loadSlot(deviceHash, 2, "Occupied");`
/// `let isOccupied = ls(deviceHash, 2, "Occupied");`
LoadSlot(
Spanned<LiteralOrVariable<'a>>,
Spanned<Literal<'a>>,
Spanned<Literal<'a>>
),
/// Stores a value of LogicType on a device by the index value
/// ## IC10
/// `ss d0 0 "Open" 1`
/// ## Slang
/// `setSlot(deviceHash, 0, "Open", true);`
/// `ss(deviceHash, 0, "Open", true);`
SetSlot(
Spanned<LiteralOrVariable<'a>>,
Spanned<Literal<'a>>,
Spanned<Literal<'a>>,
Box<Spanned<Expression<'a>>>
)
}
}
impl std::fmt::Display for System {
impl<'a> std::fmt::Display for System<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
System::Yield => write!(f, "yield()"),
@@ -235,6 +259,8 @@ impl std::fmt::Display for System {
System::SetOnDeviceBatchedNamed(a, b, c, d) => {
write!(f, "setOnDeviceBatchedNamed({}, {}, {}, {})", a, b, c, d)
}
System::LoadSlot(a, b, c) => write!(f, "loadSlot({}, {}, {})", a, b, c),
System::SetSlot(a, b, c, d) => write!(f, "setSlot({}, {}, {}, {})", a, b, c, d),
}
}
}
@@ -242,13 +268,13 @@ impl std::fmt::Display for System {
#[allow(clippy::large_enum_variant)]
#[derive(Debug, PartialEq, Eq)]
/// This represents built in functions that cannot be overwritten, but can be invoked by the user as functions.
pub enum SysCall {
System(System),
pub enum SysCall<'a> {
System(System<'a>),
/// Represents any mathmatical function that can be called.
Math(Math),
Math(Math<'a>),
}
impl Documentation for SysCall {
impl<'a> Documentation for SysCall<'a> {
fn docs(&self) -> String {
match self {
Self::System(s) => s.docs(),
@@ -264,7 +290,7 @@ impl Documentation for SysCall {
}
}
impl std::fmt::Display for SysCall {
impl<'a> std::fmt::Display for SysCall<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SysCall::System(s) => write!(f, "{}", s),
@@ -273,7 +299,7 @@ impl std::fmt::Display for SysCall {
}
}
impl SysCall {
impl<'a> SysCall<'a> {
pub fn is_syscall(identifier: &str) -> bool {
tokenizer::token::is_syscall(identifier)
}

34
rust_compiler/libs/parser/src/test/mod.rs
View File

@@ -160,3 +160,37 @@ fn test_negative_literal_const() -> Result<()> {
Ok(())
}
#[test]
fn test_ternary_expression() -> Result<()> {
let expr = parser!(r#"let i = x ? 1 : 2;"#).parse()?.unwrap();
assert_eq!("(let i = (x ? 1 : 2))", expr.to_string());
Ok(())
}
#[test]
fn test_complex_binary_with_ternary() -> Result<()> {
let expr = parser!("let i = (x ? 1 : 3) * 2;").parse()?.unwrap();
assert_eq!("(let i = ((x ? 1 : 3) * 2))", expr.to_string());
Ok(())
}
#[test]
fn test_operator_prescedence_with_ternary() -> Result<()> {
let expr = parser!("let x = x ? 1 : 3 * 2;").parse()?.unwrap();
assert_eq!("(let x = (x ? 1 : (3 * 2)))", expr.to_string());
Ok(())
}
#[test]
fn test_nested_ternary_right_associativity() -> Result<()> {
let expr = parser!("let i = a ? b : c ? d : e;").parse()?.unwrap();
assert_eq!("(let i = (a ? b : (c ? d : e)))", expr.to_string());
Ok(())
}

227
rust_compiler/libs/parser/src/tree_node.rs
View File

@@ -1,24 +1,22 @@
use std::ops::Deref;
use crate::sys_call;
use super::sys_call::SysCall;
use crate::sys_call;
use std::{borrow::Cow, ops::Deref};
use tokenizer::token::Number;
#[derive(Debug, Eq, PartialEq, Clone)]
pub enum Literal {
pub enum Literal<'a> {
Number(Number),
String(String),
String(Cow<'a, str>),
Boolean(bool),
}
#[derive(Debug, Eq, PartialEq, Clone)]
pub enum LiteralOr<T> {
Literal(Spanned<Literal>),
pub enum LiteralOr<'a, T> {
Literal(Spanned<Literal<'a>>),
Or(Spanned<T>),
}
impl<T: std::fmt::Display> std::fmt::Display for LiteralOr<T> {
impl<'a, T: std::fmt::Display> std::fmt::Display for LiteralOr<'a, T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Literal(l) => write!(f, "{l}"),
@@ -27,7 +25,7 @@ impl<T: std::fmt::Display> std::fmt::Display for LiteralOr<T> {
}
}
impl std::fmt::Display for Literal {
impl<'a> std::fmt::Display for Literal<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Literal::Number(n) => write!(f, "{}", n),
@@ -38,16 +36,16 @@ impl std::fmt::Display for Literal {
}
#[derive(Debug, PartialEq, Eq)]
pub enum BinaryExpression {
Add(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Multiply(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Divide(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Subtract(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Exponent(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Modulo(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
pub enum BinaryExpression<'a> {
Add(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Multiply(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Divide(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Subtract(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Exponent(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Modulo(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
}
impl std::fmt::Display for BinaryExpression {
impl<'a> std::fmt::Display for BinaryExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BinaryExpression::Add(l, r) => write!(f, "({} + {})", l, r),
@@ -61,19 +59,19 @@ impl std::fmt::Display for BinaryExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub enum LogicalExpression {
And(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Or(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
Not(Box<Spanned<Expression>>),
Equal(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
NotEqual(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
GreaterThan(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
GreaterThanOrEqual(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
LessThan(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
LessThanOrEqual(Box<Spanned<Expression>>, Box<Spanned<Expression>>),
pub enum LogicalExpression<'a> {
And(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Or(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
Not(Box<Spanned<Expression<'a>>>),
Equal(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
NotEqual(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
GreaterThan(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
GreaterThanOrEqual(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
LessThan(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
LessThanOrEqual(Box<Spanned<Expression<'a>>>, Box<Spanned<Expression<'a>>>),
}
impl std::fmt::Display for LogicalExpression {
impl<'a> std::fmt::Display for LogicalExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
LogicalExpression::And(l, r) => write!(f, "({} && {})", l, r),
@@ -90,25 +88,25 @@ impl std::fmt::Display for LogicalExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub struct AssignmentExpression {
pub assignee: Box<Spanned<Expression>>,
pub expression: Box<Spanned<Expression>>,
pub struct AssignmentExpression<'a> {
pub assignee: Box<Spanned<Expression<'a>>>,
pub expression: Box<Spanned<Expression<'a>>>,
}
impl std::fmt::Display for AssignmentExpression {
impl<'a> std::fmt::Display for AssignmentExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "({} = {})", self.assignee, self.expression)
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct FunctionExpression {
pub name: Spanned<String>,
pub arguments: Vec<Spanned<String>>,
pub body: BlockExpression,
pub struct FunctionExpression<'a> {
pub name: Spanned<Cow<'a, str>>,
pub arguments: Vec<Spanned<Cow<'a, str>>>,
pub body: BlockExpression<'a>,
}
impl std::fmt::Display for FunctionExpression {
impl<'a> std::fmt::Display for FunctionExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
@@ -125,9 +123,9 @@ impl std::fmt::Display for FunctionExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub struct BlockExpression(pub Vec<Spanned<Expression>>);
pub struct BlockExpression<'a>(pub Vec<Spanned<Expression<'a>>>);
impl std::fmt::Display for BlockExpression {
impl<'a> std::fmt::Display for BlockExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
@@ -142,12 +140,12 @@ impl std::fmt::Display for BlockExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub struct InvocationExpression {
pub name: Spanned<String>,
pub arguments: Vec<Spanned<Expression>>,
pub struct InvocationExpression<'a> {
pub name: Spanned<Cow<'a, str>>,
pub arguments: Vec<Spanned<Expression<'a>>>,
}
impl std::fmt::Display for InvocationExpression {
impl<'a> std::fmt::Display for InvocationExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
@@ -163,25 +161,25 @@ impl std::fmt::Display for InvocationExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub struct MemberAccessExpression {
pub object: Box<Spanned<Expression>>,
pub member: Spanned<String>,
pub struct MemberAccessExpression<'a> {
pub object: Box<Spanned<Expression<'a>>>,
pub member: Spanned<Cow<'a, str>>,
}
impl std::fmt::Display for MemberAccessExpression {
impl<'a> std::fmt::Display for MemberAccessExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}.{}", self.object, self.member)
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct MethodCallExpression {
pub object: Box<Spanned<Expression>>,
pub method: Spanned<String>,
pub arguments: Vec<Spanned<Expression>>,
pub struct MethodCallExpression<'a> {
pub object: Box<Spanned<Expression<'a>>>,
pub method: Spanned<Cow<'a, str>>,
pub arguments: Vec<Spanned<Expression<'a>>>,
}
impl std::fmt::Display for MethodCallExpression {
impl<'a> std::fmt::Display for MethodCallExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
@@ -198,12 +196,12 @@ impl std::fmt::Display for MethodCallExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub enum LiteralOrVariable {
Literal(Literal),
Variable(Spanned<String>),
pub enum LiteralOrVariable<'a> {
Literal(Literal<'a>),
Variable(Spanned<Cow<'a, str>>),
}
impl std::fmt::Display for LiteralOrVariable {
impl<'a> std::fmt::Display for LiteralOrVariable<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
LiteralOrVariable::Literal(l) => write!(f, "{}", l),
@@ -213,46 +211,46 @@ impl std::fmt::Display for LiteralOrVariable {
}
#[derive(Debug, PartialEq, Eq)]
pub struct ConstDeclarationExpression {
pub name: Spanned<String>,
pub value: LiteralOr<SysCall>,
pub struct ConstDeclarationExpression<'a> {
pub name: Spanned<Cow<'a, str>>,
pub value: LiteralOr<'a, SysCall<'a>>,
}
impl ConstDeclarationExpression {
impl<'a> ConstDeclarationExpression<'a> {
pub fn is_syscall_supported(call: &SysCall) -> bool {
use sys_call::System;
matches!(call, SysCall::System(sys) if matches!(sys, System::Hash(_)))
}
}
impl std::fmt::Display for ConstDeclarationExpression {
impl<'a> std::fmt::Display for ConstDeclarationExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "(const {} = {})", self.name, self.value)
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct DeviceDeclarationExpression {
pub struct DeviceDeclarationExpression<'a> {
/// any variable-like name
pub name: Spanned<String>,
pub name: Spanned<Cow<'a, str>>,
/// The device port, ex. (db, d0, d1, d2, d3, d4, d5)
pub device: String,
pub device: Cow<'a, str>,
}
impl std::fmt::Display for DeviceDeclarationExpression {
impl<'a> std::fmt::Display for DeviceDeclarationExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "(device {} = {})", self.name, self.device)
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct IfExpression {
pub condition: Box<Spanned<Expression>>,
pub body: Spanned<BlockExpression>,
pub else_branch: Option<Box<Spanned<Expression>>>,
pub struct IfExpression<'a> {
pub condition: Box<Spanned<Expression<'a>>>,
pub body: Spanned<BlockExpression<'a>>,
pub else_branch: Option<Box<Spanned<Expression<'a>>>>,
}
impl std::fmt::Display for IfExpression {
impl<'a> std::fmt::Display for IfExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "(if ({}) {}", self.condition, self.body)?;
if let Some(else_branch) = &self.else_branch {
@@ -263,23 +261,40 @@ impl std::fmt::Display for IfExpression {
}
#[derive(Debug, PartialEq, Eq)]
pub struct LoopExpression {
pub body: Spanned<BlockExpression>,
pub struct LoopExpression<'a> {
pub body: Spanned<BlockExpression<'a>>,
}
impl std::fmt::Display for LoopExpression {
impl<'a> std::fmt::Display for LoopExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "(loop {})", self.body)
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct WhileExpression {
pub condition: Box<Spanned<Expression>>,
pub body: BlockExpression,
pub struct WhileExpression<'a> {
pub condition: Box<Spanned<Expression<'a>>>,
pub body: BlockExpression<'a>,
}
impl std::fmt::Display for WhileExpression {
#[derive(Debug, PartialEq, Eq)]
pub struct TernaryExpression<'a> {
pub condition: Box<Spanned<Expression<'a>>>,
pub true_value: Box<Spanned<Expression<'a>>>,
pub false_value: Box<Spanned<Expression<'a>>>,
}
impl<'a> std::fmt::Display for TernaryExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"({} ? {} : {})",
self.condition, self.true_value, self.false_value
)
}
}
impl<'a> std::fmt::Display for WhileExpression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "(while {} {})", self.condition, self.body)
}
@@ -347,32 +362,33 @@ impl<T> Deref for Spanned<T> {
}
#[derive(Debug, PartialEq, Eq)]
pub enum Expression {
Assignment(Spanned<AssignmentExpression>),
Binary(Spanned<BinaryExpression>),
Block(Spanned<BlockExpression>),
pub enum Expression<'a> {
Assignment(Spanned<AssignmentExpression<'a>>),
Binary(Spanned<BinaryExpression<'a>>),
Block(Spanned<BlockExpression<'a>>),
Break(Span),
ConstDeclaration(Spanned<ConstDeclarationExpression>),
ConstDeclaration(Spanned<ConstDeclarationExpression<'a>>),
Continue(Span),
Declaration(Spanned<String>, Box<Spanned<Expression>>),
DeviceDeclaration(Spanned<DeviceDeclarationExpression>),
Function(Spanned<FunctionExpression>),
If(Spanned<IfExpression>),
Invocation(Spanned<InvocationExpression>),
Literal(Spanned<Literal>),
Logical(Spanned<LogicalExpression>),
Loop(Spanned<LoopExpression>),
MemberAccess(Spanned<MemberAccessExpression>),
MethodCall(Spanned<MethodCallExpression>),
Negation(Box<Spanned<Expression>>),
Priority(Box<Spanned<Expression>>),
Return(Box<Spanned<Expression>>),
Syscall(Spanned<SysCall>),
Variable(Spanned<String>),
While(Spanned<WhileExpression>),
Declaration(Spanned<Cow<'a, str>>, Box<Spanned<Expression<'a>>>),
DeviceDeclaration(Spanned<DeviceDeclarationExpression<'a>>),
Function(Spanned<FunctionExpression<'a>>),
If(Spanned<IfExpression<'a>>),
Invocation(Spanned<InvocationExpression<'a>>),
Literal(Spanned<Literal<'a>>),
Logical(Spanned<LogicalExpression<'a>>),
Loop(Spanned<LoopExpression<'a>>),
MemberAccess(Spanned<MemberAccessExpression<'a>>),
MethodCall(Spanned<MethodCallExpression<'a>>),
Negation(Box<Spanned<Expression<'a>>>),
Priority(Box<Spanned<Expression<'a>>>),
Return(Option<Box<Spanned<Expression<'a>>>>),
Syscall(Spanned<SysCall<'a>>),
Ternary(Spanned<TernaryExpression<'a>>),
Variable(Spanned<Cow<'a, str>>),
While(Spanned<WhileExpression<'a>>),
}
impl std::fmt::Display for Expression {
impl<'a> std::fmt::Display for Expression<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Expression::Assignment(e) => write!(f, "{}", e),
@@ -393,8 +409,17 @@ impl std::fmt::Display for Expression {
Expression::MethodCall(e) => write!(f, "{}", e),
Expression::Negation(e) => write!(f, "(-{})", e),
Expression::Priority(e) => write!(f, "({})", e),
Expression::Return(e) => write!(f, "(return {})", e),
Expression::Return(e) => write!(
f,
"(return {})",
if let Some(e) = e {
e.to_string()
} else {
"".to_string()
}
),
Expression::Syscall(e) => write!(f, "{}", e),
Expression::Ternary(e) => write!(f, "{}", e),
Expression::Variable(id) => write!(f, "{}", id),
Expression::While(e) => write!(f, "{}", e),
}

3
rust_compiler/libs/tokenizer/Cargo.toml
View File

@@ -5,9 +5,10 @@ edition = "2024"
[dependencies]
rust_decimal = { workspace = true }
quick-error = { workspace = true }
lsp-types = { workspace = true }
thiserror = { workspace = true }
helpers = { path = "../helpers" }
logos = "0.16"
[dev-dependencies]
anyhow = { version = "^1" }

953
rust_compiler/libs/tokenizer/src/lib.rs
View File

File diff suppressed because it is too large Load Diff

285
rust_compiler/libs/tokenizer/src/token.rs
View File

@@ -1,5 +1,59 @@
use std::borrow::Cow;
use helpers::prelude::*;
use logos::{Lexer, Logos, Skip, Span};
use lsp_types::{Diagnostic, DiagnosticSeverity, Position, Range};
use rust_decimal::Decimal;
use thiserror::Error;
#[derive(Debug, Error, Default, Clone, PartialEq)]
pub enum LexError {
#[error("Attempted to parse an invalid number: {2}")]
NumberParse(usize, Span, String),
#[error("An invalid character was found in token stream: {2}")]
InvalidInput(usize, Span, String),
#[default]
#[error("An unknown error occurred")]
Other,
}
impl From<LexError> for Diagnostic {
fn from(value: LexError) -> Self {
match value {
LexError::NumberParse(line, col, str) | LexError::InvalidInput(line, col, str) => {
Diagnostic {
range: Range {
start: Position {
character: col.start as u32,
line: line as u32,
},
end: Position {
line: line as u32,
character: col.end as u32,
},
},
severity: Some(DiagnosticSeverity::ERROR),
message: str,
..Default::default()
}
}
_ => Diagnostic::default(),
}
}
}
impl LexError {
pub fn from_lexer<'a>(lex: &mut Lexer<'a, TokenType<'a>>) -> Self {
let mut span = lex.span();
let line = lex.extras.line_count;
span.start -= lex.extras.line_start_index;
span.end -= lex.extras.line_start_index;
Self::InvalidInput(line, span, lex.slice().chars().as_str().to_string())
}
}
// Define a local macro to consume the list
macro_rules! generate_check {
@@ -10,29 +64,40 @@ macro_rules! generate_check {
}
}
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct Token {
/// The type of the token
pub token_type: TokenType,
/// The line where the token was found
pub line: usize,
/// The column where the token was found
pub column: usize,
pub original_string: Option<String>,
#[derive(Default)]
pub struct Extras {
pub line_count: usize,
pub line_start_index: usize,
}
impl Token {
pub fn new(
token_type: TokenType,
line: usize,
column: usize,
original: Option<String>,
) -> Self {
fn update_line_index<'a>(lex: &mut Lexer<'a, TokenType<'a>>) -> Skip {
lex.extras.line_count += 1;
lex.extras.line_start_index = lex.span().end;
Skip
}
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct Token<'a> {
/// The type of the token
pub token_type: TokenType<'a>,
/// The line where the token was found
pub line: usize,
/// The span where the token starts and ends
pub span: Span,
}
impl<'a> std::fmt::Display for Token<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.token_type)
}
}
impl<'a> Token<'a> {
pub fn new(token_type: TokenType<'a>, line: usize, span: Span) -> Self {
Self {
token_type,
line,
column,
original_string: original,
span,
}
}
}
@@ -79,25 +144,187 @@ impl Temperature {
}
}
#[derive(Debug, PartialEq, Hash, Eq, Clone)]
pub enum TokenType {
macro_rules! symbol {
($var:ident) => {
|_| Symbol::$var
};
}
macro_rules! keyword {
($var:ident) => {
|_| Keyword::$var
};
}
#[derive(Debug, PartialEq, Hash, Eq, Clone, Logos)]
#[logos(skip r"[ \t\f]+")]
#[logos(extras = Extras)]
#[logos(error(LexError, LexError::from_lexer))]
pub enum TokenType<'a> {
#[regex(r"\n", update_line_index)]
Newline,
// matches strings with double quotes
#[regex(r#""(?:[^"\\]|\\.)*""#, |v| {
let str = v.slice();
Cow::from(&str[1..str.len() - 1])
})]
// matches strings with single quotes
#[regex(r#"'(?:[^'\\]|\\.)*'"#, |v| {
let str = v.slice();
Cow::from(&str[1..str.len() - 1])
})]
/// Represents a string token
String(String),
String(Cow<'a, str>),
#[regex(r"[0-9][0-9_]*(\.[0-9][0-9_]*)?([cfk])?", parse_number)]
/// Represents a number token
Number(Number),
#[token("true", |_| true)]
#[token("false", |_| false)]
/// Represents a boolean token
Boolean(bool),
#[token("continue", keyword!(Continue))]
#[token("const", keyword!(Const))]
#[token("let", keyword!(Let))]
#[token("fn", keyword!(Fn))]
#[token("if", keyword!(If))]
#[token("device", keyword!(Device))]
#[token("else", keyword!(Else))]
#[token("return", keyword!(Return))]
#[token("enum", keyword!(Enum))]
#[token("loop", keyword!(Loop))]
#[token("break", keyword!(Break))]
#[token("while", keyword!(While))]
/// Represents a keyword token
Keyword(Keyword),
#[regex(r"[a-zA-Z_][a-zA-Z0-9_]*", |v| Cow::from(v.slice()))]
/// Represents an identifier token
Identifier(String),
Identifier(Cow<'a, str>),
#[token("(", symbol!(LParen))]
#[token(")", symbol!(RParen))]
#[token("{", symbol!(LBrace))]
#[token("}", symbol!(RBrace))]
#[token("[", symbol!(LBracket))]
#[token("]", symbol!(RBracket))]
#[token(";", symbol!(Semicolon))]
#[token(":", symbol!(Colon))]
#[token(",", symbol!(Comma))]
#[token("+", symbol!(Plus))]
#[token("-", symbol!(Minus))]
#[token("*", symbol!(Asterisk))]
#[token("/", symbol!(Slash))]
#[token("<", symbol!(LessThan))]
#[token(">", symbol!(GreaterThan))]
#[token("=", symbol!(Assign))]
#[token("!", symbol!(LogicalNot))]
#[token(".", symbol!(Dot))]
#[token("^", symbol!(Caret))]
#[token("%", symbol!(Percent))]
#[token("?", symbol!(Question))]
#[token("==", symbol!(Equal))]
#[token("!=", symbol!(NotEqual))]
#[token("&&", symbol!(LogicalAnd))]
#[token("||", symbol!(LogicalOr))]
#[token("<=", symbol!(LessThanOrEqual))]
#[token(">=", symbol!(GreaterThanOrEqual))]
#[token("**", symbol!(Exp))]
/// Represents a symbol token
Symbol(Symbol),
#[token("//", |lex| Comment::Line(read_line(lex)))]
#[token("///", |lex| Comment::Doc(read_line(lex)))]
/// Represents a comment, both a line comment and a doc comment
Comment(Comment<'a>),
#[end]
/// Represents an end of file token
EOF,
}
impl Documentation for TokenType {
fn read_line<'a>(lexer: &mut Lexer<'a, TokenType<'a>>) -> Cow<'a, str> {
let rem = lexer.remainder();
let len = rem.find('\n').unwrap_or(rem.len());
let content = rem[..len].trim().to_string();
lexer.bump(len);
Cow::from(content)
}
#[derive(Hash, Debug, Eq, PartialEq, Clone)]
pub enum Comment<'a> {
Line(Cow<'a, str>),
Doc(Cow<'a, str>),
}
fn parse_number<'a>(lexer: &mut Lexer<'a, TokenType<'a>>) -> Result<Number, LexError> {
let slice = lexer.slice();
let last_char = slice.chars().last().unwrap_or_default();
let (num_str, suffix) = match last_char {
'c' | 'k' | 'f' => (&slice[..slice.len() - 1], Some(last_char)),
_ => (slice, None),
};
let clean_str = if num_str.contains('_') {
num_str.replace('_', "")
} else {
num_str.to_string()
};
let line = lexer.extras.line_count;
let mut span = lexer.span();
span.end -= lexer.extras.line_start_index;
span.start -= lexer.extras.line_start_index;
let num = if clean_str.contains('.') {
Number::Decimal(
clean_str
.parse::<Decimal>()
.map_err(|_| LexError::NumberParse(line, span, slice.to_string()))?,
)
} else {
Number::Integer(
clean_str
.parse::<i128>()
.map_err(|_| LexError::NumberParse(line, span, slice.to_string()))?,
)
};
if let Some(suffix) = suffix {
Ok(match suffix {
'c' => Temperature::Celsius(num),
'f' => Temperature::Fahrenheit(num),
'k' => Temperature::Kelvin(num),
_ => unreachable!(),
}
.to_kelvin())
} else {
Ok(num)
}
}
impl<'a> std::fmt::Display for Comment<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Line(c) => write!(f, "// {}", c),
Self::Doc(d) => {
let lines = d
.split('\n')
.map(|s| format!("/// {s}"))
.collect::<Vec<_>>()
.join("\n");
write!(f, "{}", lines)
}
}
}
}
impl<'a> Documentation for TokenType<'a> {
fn docs(&self) -> String {
match self {
Self::Keyword(k) => k.docs(),
@@ -112,7 +339,7 @@ impl Documentation for TokenType {
helpers::with_syscalls!(generate_check);
impl From<TokenType> for u32 {
impl<'a> From<TokenType<'a>> for u32 {
fn from(value: TokenType) -> Self {
match value {
TokenType::String(_) => 1,
@@ -128,6 +355,7 @@ impl From<TokenType> for u32 {
| Keyword::Return => 4,
_ => 5,
},
TokenType::Comment(_) => 8,
TokenType::Identifier(s) => {
if is_syscall(&s) {
10
@@ -146,12 +374,12 @@ impl From<TokenType> for u32 {
7
}
}
TokenType::EOF => 0,
_ => 0,
}
}
}
impl std::fmt::Display for TokenType {
impl<'a> std::fmt::Display for TokenType<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
TokenType::String(s) => write!(f, "{}", s),
@@ -160,7 +388,9 @@ impl std::fmt::Display for TokenType {
TokenType::Keyword(k) => write!(f, "{:?}", k),
TokenType::Identifier(i) => write!(f, "{}", i),
TokenType::Symbol(s) => write!(f, "{}", s),
TokenType::Comment(c) => write!(f, "{}", c),
TokenType::EOF => write!(f, "EOF"),
_ => write!(f, ""),
}
}
}
@@ -306,6 +536,8 @@ pub enum Symbol {
Caret,
/// Represents the `%` symbol
Percent,
/// Represents the `?` symbol
Question,
// Double Character Symbols
/// Represents the `==` symbol
@@ -372,6 +604,7 @@ impl std::fmt::Display for Symbol {
Self::Asterisk => write!(f, "*"),
Self::Slash => write!(f, "/"),
Self::LessThan => write!(f, "<"),
Self::Question => write!(f, "?"),
Self::LessThanOrEqual => write!(f, "<="),
Self::GreaterThan => write!(f, ">"),
Self::GreaterThanOrEqual => write!(f, ">="),

35
rust_compiler/src/ffi/mod.rs
View File

@@ -96,9 +96,10 @@ pub fn free_docs_vec(v: safer_ffi::Vec<FfiDocumentedItem>) {
#[ffi_export]
pub fn compile_from_string(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::String {
let res = std::panic::catch_unwind(|| {
let input = String::from_utf16_lossy(input.as_slice());
let mut writer = BufWriter::new(Vec::new());
let tokenizer = Tokenizer::from(String::from_utf16_lossy(input.as_slice()));
let tokenizer = Tokenizer::from(input.as_str());
let parser = Parser::new(tokenizer);
let compiler = Compiler::new(parser, &mut writer, None);
@@ -120,7 +121,8 @@ pub fn compile_from_string(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::
#[ffi_export]
pub fn tokenize_line(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::Vec<FfiToken> {
let res = std::panic::catch_unwind(|| {
let tokenizer = Tokenizer::from(String::from_utf16_lossy(input.as_slice()));
let input = String::from_utf16_lossy(input.as_slice());
let tokenizer = Tokenizer::from(input.as_str());
let mut tokens = Vec::new();
@@ -136,34 +138,31 @@ pub fn tokenize_line(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::Vec<Ff
}
match token {
Err(ref e) => {
use tokenizer::token::LexError;
use tokenizer::Error::*;
let (err_str, col, og) = match e {
NumberParseError(_, _, col, og)
| DecimalParseError(_, _, col, og)
| UnknownSymbolError(_, _, col, og)
| UnknownKeywordOrIdentifierError(_, _, col, og) => {
(e.to_string(), col, og)
let (err_str, _, span) = match e {
LexError(LexError::NumberParse(line, span, err))
| LexError(LexError::InvalidInput(line, span, err)) => {
(err.to_string(), line, span)
}
_ => continue,
};
tokens.push(FfiToken {
column: *col as i32,
column: span.start as i32,
error: err_str.into(),
tooltip: "".into(),
length: og.len() as i32,
length: (span.end - span.start) as i32,
token_kind: 0,
})
}
Ok(Token {
column,
original_string,
token_type,
..
span, token_type, ..
}) => tokens.push(FfiToken {
column: column as i32,
column: span.start as i32,
error: "".into(),
length: (original_string.unwrap_or_default().len()) as i32,
length: (span.end - span.start) as i32,
tooltip: token_type.docs().into(),
token_kind: token_type.into(),
}),
@@ -179,8 +178,10 @@ pub fn tokenize_line(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::Vec<Ff
#[ffi_export]
pub fn diagnose_source(input: safer_ffi::slice::Ref<'_, u16>) -> safer_ffi::Vec<FfiDiagnostic> {
let res = std::panic::catch_unwind(|| {
let input = String::from_utf16_lossy(input.as_slice());
let mut writer = BufWriter::new(Vec::new());
let tokenizer = Tokenizer::from(String::from_utf16_lossy(input.as_slice()));
let tokenizer = Tokenizer::from(input.as_str());
let compiler = Compiler::new(Parser::new(tokenizer), &mut writer, None);
let diagnosis = compiler.compile();

72
rust_compiler/src/main.rs
View File

@@ -1,8 +1,5 @@
#![allow(clippy::result_large_err)]
#[macro_use]
extern crate quick_error;
use clap::Parser;
use compiler::Compiler;
use parser::Parser as ASTParser;
@@ -11,27 +8,39 @@ use std::{
io::{stderr, BufWriter, Read, Write},
path::PathBuf,
};
use thiserror::Error;
use tokenizer::{self, Tokenizer};
quick_error! {
#[derive(Debug)]
enum StationlangError {
TokenizerError(err: tokenizer::Error) {
from()
display("Tokenizer error: {}", err)
#[derive(Error, Debug)]
enum Error<'a> {
#[error(transparent)]
Tokenizer(tokenizer::Error),
#[error(transparent)]
Parser(parser::Error<'a>),
#[error(transparent)]
Compile(compiler::Error<'a>),
#[error(transparent)]
IO(#[from] std::io::Error),
}
impl<'a> From<parser::Error<'a>> for Error<'a> {
fn from(value: parser::Error<'a>) -> Self {
Self::Parser(value)
}
ParserError(err: parser::Error) {
from()
display("Parser error: {}", err)
}
CompileError(err: compiler::Error) {
from()
display("Compile error: {}", err)
}
IoError(err: std::io::Error) {
from()
display("IO error: {}", err)
}
impl<'a> From<compiler::Error<'a>> for Error<'a> {
fn from(value: compiler::Error<'a>) -> Self {
Self::Compile(value)
}
}
impl<'a> From<tokenizer::Error> for Error<'a> {
fn from(value: tokenizer::Error) -> Self {
Self::Tokenizer(value)
}
}
@@ -46,12 +55,17 @@ struct Args {
output_file: Option<PathBuf>,
}
fn run_logic() -> Result<(), StationlangError> {
fn run_logic<'a>() -> Result<(), Error<'a>> {
let args = Args::parse();
let input_file = args.input_file;
let tokenizer: Tokenizer = match input_file {
Some(input_file) => Tokenizer::from_path(&input_file)?,
let input_string = match input_file {
Some(input_path) => {
let mut buf = String::new();
let mut file = std::fs::File::open(input_path).unwrap();
file.read_to_string(&mut buf).unwrap();
buf
}
None => {
let mut buf = String::new();
let stdin = std::io::stdin();
@@ -62,10 +76,11 @@ fn run_logic() -> Result<(), StationlangError> {
return Ok(());
}
Tokenizer::from(buf)
buf
}
};
let tokenizer = Tokenizer::from(input_string.as_str());
let parser = ASTParser::new(tokenizer);
let mut writer: BufWriter<Box<dyn Write>> = match args.output_file {
@@ -75,20 +90,17 @@ fn run_logic() -> Result<(), StationlangError> {
let compiler = Compiler::new(parser, &mut writer, None);
let mut errors = compiler.compile();
let errors = compiler.compile();
if !errors.is_empty() {
let mut std_error = stderr();
let last = errors.pop();
let errors = errors.into_iter().map(StationlangError::from);
let errors = errors.into_iter().map(Error::from);
std_error.write_all(b"Compilation error:\n")?;
for err in errors {
std_error.write_all(format!("{}\n", err).as_bytes())?;
}
return Err(StationlangError::from(last.unwrap()));
}
writer.flush()?;
@@ -96,7 +108,7 @@ fn run_logic() -> Result<(), StationlangError> {
Ok(())
}
fn main() -> Result<(), StationlangError> {
fn main() -> anyhow::Result<()> {
run_logic()?;
Ok(())