Language basics

This is a quick introduction to XSharper as a scripting language, with more samples than text.

Basics

Structure

Structure of XSharper program is

<?xml version="1.0" encoding="utf-8"?>
<xsharper xmlns="http://www.xsharper.com/schemas/1.0" >
        <action id="action1">
        <action id="action2">
        <action>
        <action>
        ....
</xsharper>

Using XML immediately makes IntelliSense live in Visual Studio 2008: open .xsh file as XML, assign scheme to http://www.xsharper.com/schemas/1.0 , and Visual Studio will suggest attribute names, possible enumeration values and more.

To make developer's life easier, a few additional liberties are allowed:

For example, "Hello world" program can be written as

<Print>Hello, world!</Print>

or even

<PRINT value="Hello, world!" />

As you see above, value attribute and element text are usually interchangeable.

Variables

XSharper has concept of a variable: an object with case-insensitive name attached to it.

Variable name may contain spaces and special characters. It cannot, however, start with space, number, single/double/backquote, minus, plus, ~ or %. It can be an empty string though.

To set a variable:
 

Empty variable names are allowed:
 

Variable with names that start with % used to access environment variables:
 

To make the XML more readable, two syntaxes below are equivalent:
 

To unset a variable, skip the value attribute:
 

An attempt to access a unset variable produces an exception.

Transformations

One of the core features of XSharper is support for text transformations, which are controlled by transform (or, shortened, tr) attribute of almost all actions.

This attribute controls:

 

Default transformation is Expand, i.e. expand multi-expressions formatted as ${multi}

Another note is that transformation applies to all text attributes of an action, including the value. It's possible to explicitly exclude the value by setting verbatim="true" attribute.

 

Complete list of transformations can be obtained as
 

Multi-Expressions

To access a value of a variable, XSharper transformations are used:

 

Multi-expression is a piece of text included between the special characters ${ } (${{...}} or [[ ... ]] may be used instead, as described below ).

The word "multi-" means that several expressions may be concatenated with | and the first set value is chosen.

 

There are different types of expressions, and the expression type is determined by the first character of an expression as explained below.

Calculated expressions

If first character of an expression is = , it is interpreted using rules close to those of C#, although much more forgiving in conversions between types.

For example (int)"20d" will throw an exception in normal C#, yet will produce a double value of 20.0 in XSharper calculated expression.

 

Some notable differences from C# syntax:

Numbers

If an expression is a valid number, it is the value of the expression.
 

Quoted strings

If an expression starts with ', or ", or `, the expression is treated as string (expecting a matching character at the end).

 

Environment variables

If the first character is %, environment variable is used:
 

By default, environment variables of the current process are accessed. To access variables of the current user or machine, user: and machine: prefixes are used instead. For example:
 

Variable name

If all the previous checks fail, the value is treated as variable name. Empty variable names are also allowed:

 

<eval> action

Multiple expressions may be interpreted in sequence using action:

 

The script below makes a screenshot and saves it to scrshot.png:

 

Comments and blocks

Comments

For comments the usual XML comments <!-- --> may be used, or rem action:
 

Blocks

There are basic loops, blocks, and conditionals in XSharper. But with C# behind it, you can also use any construction C# allows.

The most primitive structure is sequence, which is just a sequence of commands:

 

Block is a special sequence, which can be followed by the standard try/catch/finally:

 

Throwing exceptions

Exceptions may be thrown as

 

If class is not specified, exception of type ScriptUserException is thrown:
 

There is a special exception type ScriptTerminateException which terminates the execution of the current script with error code. Unlike other exceptions, it is rethrown at the end of catch in blocks, making it similar to ThreadAbortedException in .NET:

 

This is useful for premature script termination, and there is shorter notation:
 

Error message can be produced too

 

Conditions and loops

If/Else

XSharper has the usual if/else construction, which has two notations.

The first XML-like notation, where else is included as a part of if element is a bit cumbersome. There is however an alternative syntax, with , yet there is no way to validate it against XSD schema.

 

There is a bunch of possible attributes in conditionals, isTrue, isFalse, isSet (if variable is set), isNotSet, isZero, isNotZero etc. Use xsharper /? if for more details.

While loop

XSharper has only a simple while loop.

 

Note the use of #lt# instead of &lt;, to make the expression more readable.

While loop may define a maximum number of loops:

 

Actually, while can automatically assign its loop counter value to a variable specified in name attribute, making the code a bit shorter:

 

There is also a break statement, to exit a loop prematurely:

 

ForEach loop

forEach loop iterates through each element of a collection, XML document or a rowset.

Syntax is <foreach variable='${enumerable}'>

 

Or <foreach name='x' in='${enumerable}'> where name is optional (and defaults to empty string):

 

foreach can also parse rowsets:
 

Subroutines and scripts

XSharper has a concept of a subroutine, that accepts 0 or more arguments and returns a single object value.

 

As seen above, subroutines may be invoked directly from the expressions.

To execute square subroutine from another script, it's possible to use include action:

 

Finally, execute another script as if it was executed from command line:

 

Isolation level

By default, subroutines have read-only access to the variables of the caller and any set or changed variables in the subroutine body are restored.

 
The code above will print:
 

This behaviour can be changed by caller, by changing attribute isolation

With isolation="none", when no variables are restored, the output will be
 

With isolation="high", the subroutine will have a totally clean variable list, w/o access to the caller variables at all:
 

Using C#

Code snippets

While XSharper has basic primitives like expressions, or if/else, or while loop, these are often not sufficient. Instead of reinventing the wheel, it is very easy to embed C# code into the script.

 

Everything inside <code> is a single class, so local variables defined in one code block are not accessible in another.

It's also possible to mix & match too by inserting actions into C# code. And even add methods to the class written in C#.

 

Some interesting notes:

However, when inserting larger pieces of C# code, perhaps copy-pasted from somewhere, XML escaping becomes rather annoying. There are two possible solutions.

CDATA section:
 

or an XML processing instruction:
 

Which can be further shortened from <?code to <?_:

 

Headers

As everything inside <code> is inserted by XSharper into a class, it's not possible to define complete classes to be shared with different <code> actions, or define namespaces.

To insert code outside of any class or execution sequence, there is a <header> action, which may be also inserted as <?header or <?h processing instruction.

 

References

To add a reference to another assembly, there is a <reference> action. Assemblies may be referenced by name (using name attribute) or by location (using from attribute)

 

or a bit shorter:

 

Parsing command line

XSharper scripts accept command line parameters. These parameters are declaratively defined, automatically parsed and values assigned to XSharper variables. The same declaration is also used to display script usage information.
 

At the beginning three parameters are defined. First two arguments will be assigned to from and to variables, correspondingly. The third parameter, which is optional, is a switch /overwrite which does not have any values following it (count="none"), and sets variable overwrite to true if specified (default is false).

If executed without parameters, because usage options are set to ifNoArguments the script outputs

 

and exits with exit code "1". Long text lines are automatically wrapped depending on console window width.

Add GUI to the mix

Command line parameters a great for something that is used often. For less frequently used tools it's too hard to read the documentation first, figure out the right combination of switches through trial and error, deal with escaping and so on.

Fortunately, with XSharper scripts GUI is just one click away (the look is completely customizeable, #/gui-param is just another XSharper script in Library ):

 

produces

Custom usage

While automatic usage generation is useful, it is also possible to replace it with completely custom text instead. For example, the script below displays custom usage text (by adding a
w/o name or switch attributes), and parameter parsing is handled separately (parameters have empty text, and thus not displayed in usage):

 

Note that using description attribute it's possible to define a user friendly name of a parameter that does not match the variable being set.

Output and text files

Print

As can be seen in previous examples, the core of XSharper output is <print> action, which outputs a string to console.

To make output a bit nicer, mimic operating system streams, and also categorize output, there are 5 output options.

Name Purpose Default direction
^out Normal output Standard output
^bold More visible output, like section names etc. Standard output
^info Optional output Standard output, unless //quiet is specified
^error Error messages Standard error
^debug Debug messages None. Debug output (can be seen with dbgview utility) if //debug switch is specified. Also copied to standard output, if //debugc switch is specified
^nul,^null No output

In many actions it's possible to specify where the output goes via outTo attribute.

 

Also the same outTo may redirect the output to a variable

 

also if variable name is preceded by +, value is appended to the variable instead of overwriting it

 

Text file may be used for output using ^# prefix. Also, append is possible by using +^# prefix. Also, after | encoding may be specified (default to UTF8 with BOM)

 

By default, print appends a new line to every string output, that may be switched off using newline="false" attribute:

 

Redirection

Output to one stream may be redirected to another:
 

Or to a file (in UTF8 w/o BOM):
 

Or can add + to append:
 

ReadText

Reading a text file into a variable is easy.

 

Or just can out it to console instead:

 

If it's known in advance that the file is in UTF8

 

or

 

Or, if the file is on the web-server:

 

Finally, a multi-expression can be used as well (although XML often better demonstrates writer's intentions)

 

WriteText

Writing a text file is just as easy

 

To save as UTF16 with byte order mark:

 

Or, can just use print instead

 

Data islands

Scripts often don't have the same level of data and logic separation as programs in "traditional" languages. And this is for a good reason: logic in scripts changes about as often as the data being processed (and if it's not true, "traditional" compiled programming language, with type validation etc, would be, arguably, a better choice).

So a scripting language needs a way to hardcode data into the script itself in text, XML, or CSV. Ideally, it should be as easy to use external files too. CSV support may be surprising if there is XML, but I often find CSV and text a lot easier to read and modify than XML because of reduced verbosity.

For example,

 

is a lot more readable than
 

Rowset

More often than not when writing scripts there is a need to deal with flat lists of objects, not object hierarchies. Could be lists of files, SQL tables and so on.

To represent this type of data, there is rowset action, which contains multiple rows in it. Representation may different, whatever is more convenient to the writer.

For example, XML using a special action to for every record, with attributes being values.

 

or "normal" XML using elements

 

or CSV using comma or other column separator

 

All these notations are equivalent. The final line prints the rowset nicely as a table:

 

Rowset may also be converted to a .NET DataTable using .ToDataTable() method.

By default columns are treated as text, but type may be explicitly specified. Also one rowset may be derived from another rowset with filtering and sorting. For example, the below prints total cost of cereal ( 2.49 * 9 = 21.41 ). For comparison, cost of meat is calculated using C# snippet with LINQ (this one requires .NET 3.0).

 

Xml document

There is also a basic support for XML data islands, to represent hierarchical structures.

With the following definition
 

The code below prints inner XML of paragraph par1
 

This prints the first bold text in par1:
 

The data can be also accessed from C# code, using the usual XML parsing tools from .NET 2.0:

 

Finally, if executed on machine with .NET3, LINQ for XML can be used:

 

Also, the data may be loaded from file instead of being specified directly:

 

There are many other possibilities, including conversions between XmlDoc/RowSet/.NET DataTable/text table/CSV format/XML etc.

Databases

XML data islands and rowsets are handy for small amount of data. Handling larger datasets requires proper database support. In particular, I needed XSharper to install & initialize MS SQL databases (any other database with a .NET client may be used with client factor class name specified through factory attribute of database action).

The samples below assume that there is a default SQLExpress installation on the machine.

Creating a database:
 

Adding a table:

 

In Oracle case, this would look like

 

Please note that each statement is executed separately as a command, "GO" command from MS SQL batches is not currently allowed. Separate SQL statements need to be executed as separate <sql> actions.

Inserting records.

The usual INSERT SQL is always an option, but I find its syntax rather difficult to read or change (only in MS SQL 2008 there are some notable improvements). Instead there is sqlinsert action, which is similar to rowset and may contain CSV inside:

 

Getting the data back

This is easy:
 

produces
 

There are many attributes of sql action that can be used, for example, to save the data to a rowset for future processing using toRowsetId attribute, and so on.

Upsert

sqlinsert is getting particularly useful for "upsert", add a record only if no record already exists, and update otherwise. T-SQL verbosity in this case just hits the roof. In XSharper only a new field keys is added, to specify which columns are used to find the existing record.

For example, change address of John Smith to "50 Blackberry Rd, Ottawa, ON", if he does exist, and insert the record otherwise:

 

Generated syntax depends on the updateMode, with simple and msSql choices doing the usual "if (exists) update() else insert()", which is potentially unsafe if used outside of transaction with serializable isolation level (but probably fine more most scripts). Of the two, msSQL is a bit faster, and generates something like

 

There are also merge option invoking the new SQL MERGE statement, requiring MS-SQL 2008. To get XSharper to autodetect the database updateMode can be set to auto.

Of course, if just update is needed, it can be done with a parameterized query:

 

Transactions

And the last but not least, transactions can be done too (using the usual TransactionScope from .NET2). Transaction commits automatically if the block exits normally, and rollbacks if leaving the block via exception

 

Executing programs

Being a batch file replacement language, XSharper of course can execute other executable files. And also in a more convenient way than traditional batch files, as output redirection and argument escaping (a major annoyance when writing Windows batch files) is handled w/o hacks and temporary files.

 

Output may be redirected to a variable or output stream

 

By default the output is expected to be text in default encoding (encoding can be changed using encoding attribute). It's also possible to redirect in binary mode, using binary="1" attribute.

There are different modes of command execution, specified via mode attribute:

comspec execute cmd /c program
batch save the shell command as a batch file, execute it, and then delete the batch file
direct the first argument is expected to be a name of an executable file
shellExecute use shellExecute, useful for direct opening of Word documents, or printing (specify verb='print') and so on
auto If verb is empty, use comspec otherwise shellexecute

Exit code may be saved to a variable via exitcodeTo attribute. By default, unless ignoreExitCode attribute is set, exception is thrown if the exit code is non-zero.

XSharper also provides assistance with argument escaping, by automatically escaping arguments with spaces inside with double quotes, and escaping " and other special characters with \ (which works fine with most applications but unfortunately not all as command line parsing in Windows is poorly standardized).

For example, the code below creates a temp batch file, and executes it with parameters, which are automatically escaped as needed. Parameters may be specified via <param> sub-element, which may also accept arrays, as demonstrated in the following example:

 

The script outputs
 

File operations

XSharper has built-in support for copy/move/delete/dir commands, which are useful for its batch file replacement functions. Support for .ZIP archives added in a similar fashion via #zipLib library.

The idea was to make group operations easy to use for simple cases, yet providing a capability to independently handle every individual file, confirm file overwrites and so on.

This is of the more complex and messier part of XSharper so a bit more explanation is due.

Algorithm

Implementation of the above functionality has a lot in common (and actually a common base class, ActionWithFilters) in its implementation.

There is a scanner of a directory tree, which filters files and directories matching a filter, and for every matching file contents of the block is executed. Hidden/System files are ignored unless hidden="1" is set.

<action filter="..." directoryFilter="..." hidden="0/1">
        user code executed FOR EVERY FILE
        This is run BEFORE the copy/move/delete/etc. operation
        <try>
                This is also user code, but its exceptions will be caught
                This code may set skip variable to 1, if the operation should not be performed.
                ...
                This is after the last user statement in try.
                If skip=0, here the actual Copy/Move/Delete etc will be executed with the provided file
        </try>
        <catch>
                Executed if the operation fails for a particular file.

                Note that this catch block may be executed if an error occured
                in preparation for the action, even before user code is executed.
        </catch>
        <finally>
                        Executed after the operation, whether it fails or succeedes
        </finally>
</action>

As this is technically a loop, action can break out of it.

What is special about the try/catch/finally block here is that <try> may be skipped, but <catch> will still catch copying exceptions.

A FileSystemInfo of the source file is passed to the block as "" and "from" variable (a prefix may may be added to variable via name attribute), which may be accessed as ${} or just $ inside XSharper expressions. If there is a destination location, it is set in "to" variable.

User-provided block decides whether to perform the command-specific operation (copy / move / delete / add to archive / extract) on the file by setting "skip" variable to true or false. If skip is false after completing the try block, the operation is executed.

Filter notation

There are two filters used for scanning directory tree, where both are optional. One filter is applied on found files, the other on found directories.

filter applies to found filenames without path. For example, for C:\Data\xxx.txt only xxx.txt will be evaluated against filter
directoryFilter applies to found directory names with full path. For example, for C:\Data\MyDir the whole string will be evaluated against filter.

Two different syntaxes may be used in the filter value:

wildcard A semicolon-separated list of masks. * = any number of any characters, ?=any single character. Mask may be prefixed with - to exclude, or optional + to include.

For example *.x??;-*.xls means all files with 3 letter extension that starts with x, except xls

pattern Normal regular expression
auto If filter starts with ^ it is considered to be pattern, otherwise wildcard.

List directory example

While I'm thinking of a better way to explain the above logic, here is an example that lists all *.XSH files in the current directory and its subdirectories, skipping ".svn" subdirectories. For every file its length is displayed:

<dir from="." filter="*.xsh" directoryFilter="-*\.svn" recursive="true">
        <print> ${=$.FullName}, Size=${=$.Length}</print>
</dir>

The following lists all files AND directories

<dir from="." sort="n" recursive="1" options="directories files">
        <print>${=$.fullname}</print>
        <noMatch>
                <print>Nothing found</print>
        </noMatch>
</dir>

sorted by name (sort="N"). Sorting order may be changed as in CMD.EXE, by specifying one or more of the letters:

N By name (alphabetic)
S By size (smallest first)
E By extension (alphabetic)
D By date/time (oldest first)
G Group directories first
A Access time
C Creation time
W Modification time (same as D)
- Prefix to reverse order

Please note that sort only applies to one directory, not to the whole tree, when listing directories recursively. This is consistent with dir behaviour in CMD.EXE

Copy or move files

Copy all files from current directory to r:\backup (as you see both try and catch may be skipped, to execute a piece of action after file copying completed)

<copy from="." to="r:\backup" recursive="true" hidden="true">
        <print>Copying ${from}=>${to}</print>
        <finally>
                <print>Done copying ${from}</print>
        </finally>
</copy>

Copy all files from current directory to r:\backup, ignoring all files with length>500

<copy from="." to="r:\backup" recursive="true" hidden="true">
        <set skip="${=$.length>500}" />
        <finally>
                <eval>
                        $skip?null:c.Print($)
                </eval>
        </finally>
</copy>

Also support for overwriting existing file is available. In overwrite attribute it can be chosen whether to overwrite existing files always, only if newer, never, or ask user.

The piece below demonstrates copying procedure, confirming overwrite of the existing files:

<copy from="." to="r:\backup" recursive="true" hidden="true" overwrite="confirm">
    <!-- Skip will be set to true if destination file already exists -->
        <if condition="${skip}">
                <print newline='false'>File '${to}' already exists. Overwrite (Y/N)? </print>
                <while>
                        <set key="${=char.ToLower(Console.Read())}" />
                        <if condition="${=$key=='y'}">
                                <set skip="false" />
                                <break />
                        </if>
                        <if condition="${=$key=='n'}">
                                <break />                              
                        </if>
                </while>
        </if>

        <try>
                <if isNotTrue="${skip}">
                        <print newLine="false">Copying ${from} => ${to} .... </print>
                        <sleep timeout="500" />
                </if>
        </try>
       
        <catch>
                <print outTo="^error">Ignoring ${=c.CurrentException.Message} when dealing with ${from}</print>
        </catch>
        <finally>
                <eval>
                        $skip?null:c.writeline('done');
                </eval>
        </finally>
</copy>

Deleting files

This one just follows the pattern. There are two additional attributes. deleteReadOnly and deleteRoot, which control whether readonly files, and the root directory specified in from attribute, will be deleted.

Deleting a non-existing file or directory is not an error.

For example, the below deletes r:\backup.old, including hidden, system and read only files, printing the names of deleted files.

<delete from="r:\backup.old"
                        deleteRoot="false" deleteReadonly="true" hidden="true" recursive="true">
        <print>Deleting ${from}</print>
</delete>

Creating and unpacking ZIP archives

Again, same pattern. To ZIP r:\backup folder to a zip with default compression and password 'password' run

<zip from="r:\backup" to="x.zip" recursive="true" hidden="true" password='password'>
        <print>Archiving ${}</print>
</zip>

There are two inconvenient moments with an ancient format of ZIP, which should be kept in mind:

Unzipping the archive previously created is easy too:

<delete from="r:\tmp" recursive="true" hidden="true" />
<unzip from="x.zip " to="r:\tmp" hidden="true" password='password'>
        <print>Extracting ${from} => ${to}</print>

        <!-- Ignore errors -->
        <catch />
</unzip>

Downloads

Another common need in scripting is to download a file from somewhere, and the file can be a binary or text, and it can be just saved to a HDD, or processed first (ideally w/o any temporary files).

There are many ways to do it XSharper.

A dedicated download command, saving to a file.
 

Via a dedicated download command, saving to a variable as text:

 

ftp and http are supported, including their ftps:// and https:// variations. There is also a special scheme ftpa:// meaning "active ftp", as opposed to default passive ftp.

While download action is useful, it may be sometimes more to use URLs for remote files in the same way as local filenames are used.
 

Regular expressions

Dealing with regular expressions is a very essential part of many text parsing operations. XSharper regex action is a wrapper around Regex .NET class, and is a block that executes once for every match, setting captures as variables prefixed with name attribute (if empty, and capture name is numeric, underscore is added ):

 

By default search uses IgnoreCase option. But options can be changed:

 

A piece of code may be executed if no matches were found:

 

Captures

Captures are available inside the regex block as variables. For captures without name, _1 variable is set for first capture, _2 for second, etc. _0 variable is set to the found string.

 

Output is below, demonstrating that by default capture variables are not propagated beyond the regex body:
 

To get regex just to set capture variables, setCaptures may be set. Additional attributes define that the loop should exit after the first match is found, and that all capture variables should get 'x:' prefix:

 

Output is now
 

Replacement

replace attribute can be used to replace text
 

Replace with back-reference:

 

Note that the result is unexpected, because ${1} is expanded to '1' before being passed to regular expression. Can deal with this issue using a temp variable:

 

or by using a different escape sequence:

 

or by using an internal expression:

 

or by just using C#-like syntax:
 

Miscellaneous

Controlling NT services

A small convenience feature, to start/stop NT services.

 

Delaying and timing script execution