argparse and the Argument Parser

sys.argv Calls

Using the mock_calls list from mock can be useful in figuring out how an object is being used.

parser = argparse.ArgumentParser()
parser.add_argument('--debug', action='store_true')
parser.add_argument('-d')

sys = MagicMock()
with patch('sys.argv', sys):
    args = parser.parse_args()
for item in sys.mock_calls:
    print item
print args

call.__getitem__(slice(1, None, None))
call.__getitem__().__iter__()
call.__getitem__().__getitem__(slice(0, None, None))
call.__getitem__().__getitem__().__iter__()
call.__getitem__().__getitem__().__len__()
Namespace(d=None, debug=False)

The getitem and slice

The first thing to note is the __getitem__ calls. According to the documentation it is:

Called to implement evaluation of self[key]. For sequence types, the accepted keys should be integers and slice objects.

So it looks like it is first using the built-in slice function to get a particular argument. According to the documentation the arguments are the same as for the range function (start, stop, step).

So it looks like it is doing the equivalent of [1:] in the first slice:

test = [0,1,2]

# what does it do?
print test.__getitem__(slice(1, None, None))

# are they the same?
print test[1:] == test.__getitem__(slice(1, None, None))

[1, 2]
True

One thing to note is that slice(1) is not the same thing as slice(1, None, None):

print slice(1)
print slice(1, None, None)

slice(None, 1, None)
slice(1, None, None)

Trying a lambda

So, if I give __getitem__ a function to return the arguments I want, will this work?

sys.__getitem__ = lambda x,y: ['--debug']
with patch('sys.argv', sys):
    args = parser.parse_args()
for item in sys.mock_calls:
    print item

print args

call.__getitem__(slice(1, None, None))
call.__getitem__().__iter__()
call.__getitem__().__getitem__(slice(0, None, None))
call.__getitem__().__getitem__().__iter__()
call.__getitem__().__getitem__().__len__()
Namespace(d=None, debug=True)

It looks like it does, but would it be better to just make argv a list?

argv as a list

args = ['--debug']

def getitem(index):
    return args[index]

# make a new mock since I set __getitem__ to a  lambda function
sys = MagicMock()
sys.__getitem__.side_effect = getitem

with patch('sys.argv', sys):
    parsed_args = parser.parse_args()

for item in sys.mock_calls:
    print item
print parsed_args

call.__getitem__(slice(1, None, None))
Namespace(d=None, debug=False)

It now does not make the other calls and it also does not set the debug to True, so it did not work.

But I seem to have forgotten my earlier slice check -- it is starting at the second item. I think that normally the name of the program is the first thing passed in so maybe there needs to be an extra (first) entry to simulate the command name.

Adding a Command Name

args = 'commandname --debug'.split()

def getitem(index):
    return args[index]

sys.__getitem__.side_effect = getitem

with patch('sys.argv', sys):
    parsed_args = parser.parse_args()

for item in sys.mock_calls:
    print item
print parsed_args

call.__getitem__(slice(1, None, None))
call.__getitem__(slice(1, None, None))
Namespace(d=None, debug=True)

It looks like it worked, and all but the first two calls went away, so it perhaps they were a result of me using the mock, not a normal part of the way parse_args works.

The Whole Thing

Okay, but what about the option -d?

args = 'commandname -d cow --debug'.split()

def getitem(index):
    return args[index]

sys.__getitem__.side_effect = getitem

with patch('sys.argv', sys):
    try:
        parsed_args = parser.parse_args()
    except Exception as error:
        print error

for item in sys.mock_calls:
    print item
print parsed_args

call.__getitem__(slice(1, None, None))
call.__getitem__(slice(1, None, None))
call.__getitem__(slice(1, None, None))
Namespace(d='cow', debug=True)

Well, that was kind of painful. On the one hand, I got it to work, on the other hand, I do not really know what the slice is doing since it seems to slice the same items over and over. I think, looking at the first set of calls, after the initial slice it manipulates the sliced copy and since I am passing a real list instead of a mock, the calls are now hidden.

Looking at the Code

I downloaded the python 2.7 code and looked in argparse.py and found this:

def parse_args(self, args=None, namespace=None):
    args, argv = self.parse_known_args(args, namespace)

There is more to that function, but since it is calling parse_known_args I jumped to it:

def parse_known_args(self, args=None, namespace=None):
    if args is None:
        # args default to the system args
        args = _sys.argv[1:]

Once again there is more code after that, but this explains the slice that is seen in the calls.

Later on it calls:

namespace, args = self._parse_known_args(args, namespace)

So jumping to _parse_known_args:

arg_strings_iter = iter(arg_strings)
for i, arg_string in enumerate(arg_strings_iter):

which I think explains the __iter__ call in the first set of calls. I tried stepping through the code with pudb but could only find one slice, I am not sure what the other calls were for. I suppose it would have been smarter to look at the source code first, but this is about figuring out how to use mock so I think it was helpful to try it empirically first. No fair peeking in the back of the book until you have tried at least once.