News‎ > ‎

...and then IEEE Intelligent Systems !

posted Mar 23, 2016, 5:10 AM by Alberto Bartoli
Our last post was about acceptance on IEEE TKDE of our work on automatic regex construction and ended as follows:

We are crossing our fingers as we hope to make another announcement soon...

And now, yes, we have just received another acceptance notification from a very prestigious journal: IEEE Intelligent Systems. So prestigious that their automated response to our submission was, in a sense, discouraging us of even trying: "please bear in mind that our current acceptance rate for unsolicited manuscripts is under 10%, and that we have a backlog that makes it impossible for us to accept everything we might want to, even if it is of high quality.".

This work provides a very convincing demonstration of the power and novelty of our regex-construction tool:

Building a regular expression involves a considerable amount of skill, expertise and creativity. In this work we investigate whether a machine may surrogate these qualities and construct automatically regular expressions for tasks of realistic complexity.

We discuss a large scale experiment involving more than 1700 users on 10 challenging tasks. We compared the solutions constructed by these users to those constructed by a tool based on Genetic Programming that we have recently developed and made publicly available. 

The quality of automatically-constructed solutions turned out to be similar to the quality of those constructed by the most skilled user group; and, the time for automatic construction was similar to the time required by human users.

Key findings:
  • 1,764 users: 44% novice, 38% intermediate, and 18% experienced.
  • 12 tasks.
  • On average, our tool delivered solutions with accuracy (F-measure) almost always greater than or equal to the one obtained by each category of  users.
  • On average, the time required by our tool was almost always smaller than the time required by human operators.
  • We believe these results are remarkable and highly significant. Indeed, we are not aware of any similar tool exhibiting such human-competitive performance indexes. 
  • We believe that this result is a further demonstration of the practical capabilities of Genetic Programming techniques even on commodity hardware.
Comments