Colm Networks

Additional Target Languages Return to Ragel 7

Fri, 18 May 2018 00:00:00 -0400

It was previously reported that Ragel 7 will support only a small set of host languages. I'm happy to report that support for extra host languages has been added back to Ragel 7.

When I dropped support I had an open problem in the form of managing code cloning in multiple fronteneds. Since ragel is embedded in a host language, there is a need for host-language specific parsing. Developing a frontend for each host language was a real problem and I could not see a suitable solution. It became too much to manage.

With some new advances in Colm I'm able to now combine grammar fragments and reduction actions to compose frontends using a common ragel grammar and a small set of rules specific to the frontend. This makes maintenace of the additional frontends as new features are added much simpler and allows me keep this aspect of the project going.

Changes to Ragel in Response to the Cloudflare Incident

Tue, 28 Feb 2017 00:00:00 -0500

Intro

If you’re reading this, then you’re likely familiar with the Cloudflare incident that was disclosed a few days ago. A lot has been written about the problem, following a very detailed blog post by Cloudflare. Unfortunately, due to the initial wording of that blog post, there was a lot of blame placed on Ragel by people who are unfamiliar with Ragel and the realities of using it to produce parsers.

Ragel fgoto Command

Ragel is a state machine compiler. It helps you specify a state machine using regular language constructs. In the Ragel model it is possible to jump from sub-sections of a state machine with an fgoto command in an action. This alters the current state. This is outside of the regular lagnuage model and is something you do once in a while.

An fgoto command does not affect the input increment operation. When you issue an fgoto, processing resumes on the next input character. If you want this, then you just use fgoto. If you want to resume processing on the current character you issue an fhold first. If you want to resume somewhere else, you can as well.

Ragel Error Actions

Ragel allows the programmer to embed actions into a state machines. These execute in various circumstances. Actions are blocks of code written in the host language (in this case C). The Cloudflare incident involved error actions. Error actions execute when the state machine cannot continue normal processing. They are executed in two distinct cases.

Failure occurs on the current character.
Failure occurs at EOF (not terminating in a final state).

Combining Error Actions and fgoto Command

In the first error action case it is okay to not issue an fhold before fgoto. In the second case it is not. Since we are already past the end of the buffer, we must fix the input. This is true for any kind of action that executes in the EOF case.

This requirement is where things went wrong. There was a missing fhold before the fgoto, however, since Cloudflare was never telling Ragel where the end of the input was, the action was not executing in the EOF case.

Cloudflare would have heavily tested the buggy error action when it executed in the first case and it would have worked just fine. However, turn on the second case and that’s where the problem arises.

The fact that error actions execute in two distinct cases is certainly a part of working with Ragel that requies some attention. A single construct has two contexts of sort, and it’s possible to get a false sense of security when your code works okay in one of those contexts, the more commonly occuring one.

This is part of Ragel’s design that I have debated over the years. However, I maintain that this design is better than the alternative, which would be for a parsing error handler to not execute because it occurred on EOF and not on an input character. It is better for “error action” to cover all senses of “error”.

Changes to Ragel

What can be done here without really altering existing programs much is to enhance the generated fgoto code in the context of EOF actions. Such a change would catch failures to fixup the input pointer when it is vital and forgotten.

The alternative is a warning on a lack of fhold when an fgoto is present in an EOF action, however there could be no static guarantee that the fhold was correctly placed and it would be possible for bad code to make it through with no warning.

Ragel will Target C, C++ and ASM Only

Sun, 31 Jul 2016 00:00:00 -0400

Unfortunately I must report that after release 7.0.0.9, Ragel will target C, C++ and ASM languages only.

Ragel is now developed by Colm Networks, a network security business dedicated to producing best-in-class network security solutions. While eliminating previous code generation initiatives is regretable, it is necessary for sustaining the project. Focusing our open source development efforts in areas that support our business will allow us to ensure Ragel remains a high-quality tool for the target languages it supports.

Third Development Version of Ragel 7 Released

Mon, 11 Jul 2016 00:00:00 -0400

The third development version of Ragel 7 has been released. See the first development version news item for an overview of the changes. Additional information about dependencies here.

Grab it here: ragel-7.0.0.8.tar.gz.

Dependencies:
colm-0.13.0.4.tar.gz
Optional Deps:
ragel-6.9.tar.gz
kelbt-0.16.tar.gz

Second Development Version of Ragel 7 Released

Thu, 28 Jan 2016 00:00:00 -0500

The second development version of Ragel 7 has been released. See the previous development version news item for an overview of the changes.

Grab it here: ragel-7.0.0.6.tar.gz.

Kelbt and Ragel are now optional dependencies. Building with those packages adds the original Ragel 6 parser. It becomes accessible with the option --kelbt-frontend.

In Ragel 7 colm is a mandatory dependency. It is used to build two new ragel parsers. The default is a reduction-based parser, very similar to the original ragel+kelbt parser. The alternate is recursive-descent load of Colm parse trees. It is much slower and still used for non-C languages. It will be phased out eventually.

Deps:
ragel-6.9.tar.gz
kelbt-0.16.tar.gz
colm-0.13.0.3.tar.gz

First Development Version of Ragel 7 Released

Sat, 26 Sep 2015 00:00:00 -0400

The first development version of Ragel 7 has been released. There are some major changes packed into this release.

A condition implementation overhaul. Rather than augment the input char with the result of condition evaluation, the input char transition is found, then the condition is evaluated and the transition further refined. This is conditions done right.
An intermediate codegen language. It is now possible to add languages without understanding ragel internals. All that is needed is a translation from a simple intermediate psudo-code.
NFA features for those hard-to-parse constructs and giant unions. (Supported by Barracuda Networks Inc.)

Target Languages:

C	C++	Obj-C	C#
ASM	D	Go	Java
Ruby	OCaml	Crack	Rust
Julia

Grab it here: ragel-7.0.0.5.tar.gz.

You'll need these dependencies:
ragel-6.9.tar.gz
kelbt-0.16.tar.gz
colm-0.13.0.2.tar.gz

Barracuda Networks Supports Ragel Development

Wed, 21 Jan 2015 15:15:22 -0500

Barracuda Networks, Inc. (NYSE: CUDA), a world leader in network security and storage solutions, is generously supporting development of Ragel. Improvements to Ragel will target Barracuda applications, but are also more widely applicable and represent general advancements to the state machine compiler. Thank you to Barracuda for contributing to the open source community!

Ragel, Kelbt and Colm Files Back Online

Fri, 28 Nov 2014 16:14:32 -0500

I'm happy to report that all the previous releases of ragel and others are now back online. They got left behind in the move to the colm.net domain. You can find them here: /files/.

Ragel Now Maintained by Colm Networks

Fri, 24 Oct 2014 12:23:33 -0400

As of October 2014, Ragel will be maintained by Colm Networks. This is a new consulting company founded by Dr. Adrian D. Thurston.

Since we cannot operate in the open, the git repository for Ragel will no longer be available. The project will be published as release (and pre-release) tarballs only. On the upside, Ragel will get much more attention.

The license will remain the same: GPLv2 with an exception for the generated code derived from Ragel source.