UrlFetchRpcHandler error

[smorodsky]

I used in the project slightly modifying class UrlFetchRpcHandler (file /appengine/standard/urlfetch/async/rpc.py)
Approximately once every 100,000 requests to receive an unexpected error:

UnboundLocalError: local variable 'result' referenced before assignment

It looks like the problem of garbage collection. Completely solve the problem by adding at the beginning define of the line

global result

I hope it will be useful.

[theacodes]

Interesting. Do you happen to have a full stacktrace that you're willing to share with us?

[smorodsky]

Sorry for the delay, please:

local variable 'result' referenced before assignment
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/e~my-app/30.394225512022207104/my-app.py", line 250, in get
    self.do_req(url, headers)
  File "/base/data/home/apps/e~my-app/30.394225512022207104/my-app.py", line 300, in do_req
    if result.status_code == 200:
UnboundLocalError: local variable 'result' referenced before assignment

[theacodes]

Do you mind sharing the do_req function's source with me?

[smorodsky]

def do_req(self, url, headers):
    global result
    rpc = urlfetch.create_rpc(deadline = 9)
    urlfetch.make_fetch_call(rpc, url, headers = headers)
    try:
      result = rpc.get_result()
    except Exception as e:
      logging.error(type(e).__name__ + ': ' + str(e))
      return
    if result.status_code == 200: # <-- UnboundLocalError here
      return result.content
...

[theacodes]

@smorodsky I wrote more tests in #457 but I still can't reproduce this.

[smorodsky]

I made 200,000 calls google.com and did not get a single shot. The server to which real requests are responsible slower. Obviously, this increases the likelihood of failure. Errors may be missing a few hours and then go to series.

If it would be helpful, I can add you to a real project. It consistently repeat this error.

[theacodes]

Sure - that might be helpful. It's strange though because the test cases I added should cover all failure paths.

[smorodsky]

Excuse me, I'm an idiot.

In the original code, there is another block except, and it skipped return. I have many times check it this function, but i can not understand for what reason did not notice the bug. I'm really sorry, that gave rise to worry. Thank you.

[theacodes]

@smorodsky don't sweat it! At least I got some shiny new tests out of it. :)