Date: 2012-07-09 16:31:33
From: jeni@jenitennison.com
Hi,
I have a query which is performing very poorly in OWLIM-SE, but very 
well in OWLIM-Lite. This is a simplified version which displays the same 
characteristics:
PREFIX task: <http://www.legislation.gov.uk/def/task/>
SELECT ?process (COUNT(?task) AS ?count)
WHERE {
{
SELECT DISTINCT ?task ?process
WHERE {
?task task:process ?process .
FILTER (
?process = 
<http://www.legislation.gov.uk/id/process/prepare/effects> ||
?process = 
<http://www.legislation.gov.uk/id/process/review/prepare/effects>
)
OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
FILTER (!BOUND(?unassignedEnded))
}
}
}
GROUP BY ?process
This takes several seconds in OWLIM-SE but only about 100ms with 
OWLIM-Lite (there are other differences in the servers of the two 
repositories, but only ones that would lead me to imagine that the 
OWLIM-SE machine would be faster).
To give you an idea of size, the results are:
<http://www.legislation.gov.uk/id/process/prepare/effects> 906
<http://www.legislation.gov.uk/id/process/review/prepare/effects> 907
The lines which seem to be causing the problem are
OPTIONAL { ?task task:endedBy|task:endedAt ?unassignedEnded . }
FILTER (!BOUND(?unassignedEnded))
I have tried several rewrites of these lines, including using FILTER ( 
NOT EXISTS { ... } ). This is the most performant that I've found, but 
it's still too slow. If I remove these lines, the results are:
<http://www.legislation.gov.uk/id/process/prepare/effects> 2594
<http://www.legislation.gov.uk/id/process/review/prepare/effects> 2594
so it's having to do a bit of filtering, but it's not like there are 
tens of millions of tasks for it to exclude.
Does anyone have any suggestions for how I might rewrite the query to 
get better performance, or why there might be this big difference in 
performance between versions of OWLIM?
Thanks,
Jeni

asked 03 Apr '13, 09:56

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%


Date: 2012-07-10 12:51:02
From: barry.bishop@ontotext.com
Thanks for this, Jeni.
It sounds like a query optimisation problem similar to what has occurred 
recently.
The developers are investigating now and I will report back asap.
Thanks for your patience,
barry
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-09 22:07:52
From: barry.bishop@ontotext.com
Hi Jeni,
I suspect you have run in to a recent query optimisation problem, but to 
be sure, could you tell me which version of OWLIM-SE you are using 
(including build number)?
I'll check with the developers in the morning.
Regards,
barry
link

answered 03 Apr '13, 09:56

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-09 22:26:43
From: barry.norton@ontotext.com
Just to throw in - you've tried this with a UNION rather than the 
property path alternative, I guess?
Barry2
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-10 09:48:19
From: jeni@jenitennison.com
Hi Barry2,
I hadn't, but I just did and it had no discernible effect.
Thanks,
Jeni
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-10 16:49:07
From: barry.bishop@ontotext.com
Hi Jeni,
This is indeed a bug in OWLIM-SE. It is quite specific and occurs only 
when a FILTER contains two or more expressions OR'd together and these 
expressions contain equalities.
The problem causes the OWLIM specific query optimisation step to abort 
and default back to the (unoptimised) Sesame query evaluation steps. 
This is why you see a degradation in performance to something worse than 
OWLIM-Lite (which uses the Sesame evaluation mechanism, but with 
optimisation).
We are planning a new version of OWLIM-SE very soon, originally intended 
to include Sesame 2.6.7 (which has other query optimisation fixes). 
However, we have detected an unrelated bug in the SPARQL parser in this 
version of Sesame and so we are working now to release Sesame 2.6.8 at 
the end of this week, OWLIM next week.
The question now is, how problematic is this current problem for you? 
Can you wait until next week for a new release of OWLIM?
All the best and thanks for reporting this,
barry
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-10 22:35:11
From: jeni@jenitennison.com
Hi Barry,
Thank you for investigating. That explains a lot of my current performance woes :)
So long as we know a fix is coming next week, I think we'll be able to cope, but just to explore other options so that we can present them to our customer: do you know when the bug was introduced? How many versions back would we have to revert, while waiting for the next release, to avoid it?
Thanks,
Jeni
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-11 06:01:38
From: marek_surek@yahoo.co.uk
Hi,
I experience the similar issues. The build b5123(OWLIM 5.0) was last build which didn't take advantage of new QueryOptimizer and it gave me expected performance. The new query optimizer(which causes the problems) seems to be fully functioning from build b5183 (at least it was the point where my query issues started to appear). I think there won't be much difference between your version (b5208) and b5123 as long as it suppose to be maintaince release mostly aimed on bugfixing.
Best regards,
Marek
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Date: 2012-07-10 09:47:52
From: jeni@jenitennison.com
Hi Barry,
It's OWLIM-SE 5.1 release 5208.
Thanks,
Jeni
link

answered 03 Apr '13, 09:57

Discussion-Board-Archive's gravatar image

Discussion-B...
6.1k133156205
accept rate: 30%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×261
×243
×38

Asked: 03 Apr '13, 09:56

Seen: 862 times

Last updated: 03 Apr '13, 09:57

powered by BitNami OSQA