<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN">
<html><body style='font-size: 14pt; font-family: Verdana,Geneva,sans-serif'>
<p>Hi, yes it is possible, scanners execute in parallel, however you don't get all the guarantees you normally get when you simply union. There may be some backtracking and leaving actions don't always execute. This second issue is something I'm hoping to fix with ragel 8.</p>
<p>If you want to do things in parallel as is the case with a normal union, you can scan using the ** operator. This is a kleene star that favours staying in the machine over starting to match a new token. If that doesn't work for you, you can implement the scanning functionality yourself, just using ragel to match a single token from the current starting location. Simply union all your patterns and wrap that in a loop.</p>
<p>On 2018-12-16 15:33, Filipp Andronov wrote:</p>
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div dir="ltr">Good day,
<div> </div>
<div>Please help me to understand if the following thing is possible with using ragel. I have a database of UTF-8 patterns, which I've compiled in ragel <span class="gmail-il">machine</span> with using a bit of python magic.</div>
<div> </div>
<div>From that compiled databse Im able to build a scanner:</div>
<div>main := |*</div>
<div> Database => {issue_token; fbreak}</div>
<div> any; # ignore unmatched</div>
<div>*|</div>
<div> </div>
<div>Having a stream of recognized tokens, I'd like to calculate some properties of each token, like how many runes are inside, different Unicode Scripts bounds and Class bounds. Right now Im doing that by a separate scan of found tokens text, but could it be done better and embbed inside scanner itself?</div>
<div> </div>
<div>For each property I could easealy generate FSM from UnicodeData files but have no idea how to compose machines to run in parallel?</div>
<div> </div>
<div>Idially a such combined <span class="gmail-il">machine</span> would analyse input stream and call actions like:</div>
<div>- PatternStart</div>
<div>- New rune</div>
<div>- New Class</div>
<div>- New rune</div>
<div>- New Class</div>
<div>- New Script</div>
<div>- PatternFound</div>
<div> </div>
<div>But some other approximation is fine too :)</div>
<div> </div>
-- <br />
<div class="gmail_signature" dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">Thanks and regards,<br /><br />Philip Andronov<br />------------------------------------------<br />Leading software engineer,
<div><a href="http://mail.ru">mail.ru</a></div>
<div>
<div> </div>
</div>
</div>
</div>
</div>
</div>
</div>
<br />
<pre>_______________________________________________
ragel-users mailing list
<a href="mailto:ragel-users@colm.net">ragel-users@colm.net</a>
<a href="http://www.colm.net/cgi-bin/mailman/listinfo/ragel-users">http://www.colm.net/cgi-bin/mailman/listinfo/ragel-users</a>
</pre>
</blockquote>
</body></html>