bug: OXFProperties.getPropertySet calls url.openConnection() every time

For 29,062 invocations of a standalone pipeline (ie no calls to other pipelines within) I'm seeing 400,824 calls to java.net.URLConnection.getLastModified() !

Most (93%) of these of from OXFProperties.getPropertySet - the time spent isn't significant because in my case it's a local file: but if this was a remote url this would be a real problem.

Looking at OXFProperties, it has code to try and restrict the last modified check to every 5 seconds. This check looks broken though, because the lastUpdate variable is only updated if the file has actually changed:

    final long current = System.currentTimeMillis();
   
    if( lastUpdate + RELOAD_DELAY  >= current ) break done;
   
    final java.net.URL url = URLFactory.createURL( key );
    final java.net.URLConnection uc = url.openConnection();
   
    if ( propertyStore != null && uc.getLastModified() <= lastUpdate ) break done;

    ...

    lastUpdate = current;

This means if the properties file hasn't changed for 5 seconds, it gets checked every time, which I don't think was the intention. It needs to be updated every time uc.getLastModified is called:

    if ( propertyStore != null && uc.getLastModified() <= lastUpdate ) {
        lastUpdate = current;  
        break done;
    }

Adrian

Adrian,

That seems reasonable, and that's a great catch! I made the change but haven't committed it yet.

Have you noticed a change in the number of calls to java.net.URLConnection.getLastModified() and/or performance after this change? Have you also tried to check that modified properties still reload?

-Erik

Adrian Baker wrote:

For 29,062 invocations of a standalone pipeline (ie no calls to other pipelines within) I'm seeing 400,824 calls to java.net.URLConnection.getLastModified() !

Most (93%) of these of from OXFProperties.getPropertySet - the time spent isn't significant because in my case it's a local file: but if this was a remote url this would be a real problem.

Looking at OXFProperties, it has code to try and restrict the last modified check to every 5 seconds. This check looks broken though, because the lastUpdate variable is only updated if the file has actually changed:

    final long current = System.currentTimeMillis();
       if( lastUpdate + RELOAD_DELAY  >= current ) break done;
       final java.net.URL url = URLFactory.createURL( key );
    final java.net.URLConnection uc = url.openConnection();
       if ( propertyStore != null && uc.getLastModified() <= lastUpdate ) break done;

    ...

    lastUpdate = current;

This means if the properties file hasn't changed for 5 seconds, it gets checked every time, which I don't think was the intention. It needs to be updated every time uc.getLastModified is called:

    if ( propertyStore != null && uc.getLastModified() <= lastUpdate ) {
        lastUpdate = current;          break done;
    }

Adrian


------------------------------------------------------------------------


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws