Python subprocess "object has no attribute 'fileno'" error

This code generates "AttributeError: 'Popen' object has no attribute 'fileno'" when run with Python 2.5.1

Code:

def get_blame(filename): 
    proc = []
    proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
    proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)
    proc.append(Popen(['tr', r"'\040'", r"';'"], stdin=proc[-1]), stdout=PIPE)
    proc.append(Popen(['cut', r"-d", r"\;", '-f', '3'], stdin=proc[-1]), stdout=PIPE)
    return proc[-1].stdout.read()

Stack:

function walk_folder in blame.py at line 55
print_file(os.path.join(os.getcwd(), filename), path)

function print_file in blame.py at line 34
users = get_blame(filename)

function get_blame in blame.py at line 20
proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)

function __init__ in subprocess.py at line 533
(p2cread, p2cwrite,

function _get_handles in subprocess.py at line 830
p2cread = stdin.fileno()

This code should be working the python docs describe this usage.

From stackoverflow

• looks like syntax error. except first append the rest are erroneous (review brackets).

  epochwolf : It's not actually.

  SilentGhost : it's not what? every single answer included the fact that you have a problem with your brackets! it is a syntax issue, on top of piping of course.

  epochwolf : It's a syntax issue but not a syntax error oddly enough. The code is directly pasted from the script I had. The error was the error that was generated. A syntax error would have killed the script in the middle of the function definition. The function actually gets called. I think it's because list.append() accepts multiple arguments.

  SilentGhost : no it doesn't. append is never called, because error occurs at the initialisation of Popen, which is fairly obvious from your error stack. so yes it is a syntax error on your part. no SyntaxError raised, but a syntax issue nevertheless.

  epochwolf : So it is... interesting. I've been doing way too much compiled stuff lately. I'm forgetting how dynamic python is.

• You want the stdout of the process, so replace your stdin=proc[-1] with stdin=proc[-1].stdout

  Also, you need to move your paren, it should come after the stdout argument.

   proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)
  

  should be:

   proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1].stdout, stdout=PIPE))
  

  Fix your other append calls in the same way.

• Three things

  First, your ()'s are wrong.

  Second, the result of subprocess.Popen() is a process object, not a file.

  proc = []
  proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
  proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1]), stdout=PIPE)
  

  The value of proc[-1] isn't the file, it's the process that contains the file.

  proc.append(Popen(['tr', '-s', r"'\040'"], stdin=proc[-1].stdout, stdout=PIPE))
  

  Third, don't do all that tr and cut junk in the shell, few things could be slower. Write the tr and cut processing in Python -- it's faster and simpler.

  Douglas Leeder : Was the 'Tree' for humour - or did you mean "Three"?

  S.Lott : Thanks for the comment. It's fixed.

  epochwolf : The script will only be used on linux. It's more of a one off tool than anything else. Using shell tools was easier than trying to figure out the python equivalent. Just dropping in a working shell command is easier than debugging regex.

  S.Lott : There's no complex regex required to parse the svn blame output. Using shell scripts is -- in the long run -- MORE complex because the shell language is such a collection of random features. Python is -- in the long run -- simpler.

  epochwolf : In the long run of course. I was converting a bash script that didn't work very well into python.

• There's a few weird things in the script,

  • Why are you storing each process in a list? Wouldn't it be much more readable to simply use variables? Removing all the .append()s reveals an syntax error, several times you have passed stdout=PIPE to the append arguments, instead of Popen:

    proc.append(Popen(...), stdout=PIPE)
    

    So a straight-rewrite (still with errors I'll mention in a second) would become..

    def get_blame(filename): 
        blame = Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE)
        tr1 = Popen(['tr', '-s', r"'\040'"], stdin=blame, stdout=PIPE)
        tr2 = Popen(['tr', r"'\040'", r"';'"], stdin=tr1), stdout=PIPE)
        cut = Popen(['cut', r"-d", r"\;", '-f', '3'], stdin=tr2, stdout=PIPE)
        return cut.stdout.read()
    

  • On each subsequent command, you have passed the Popen object, not that processes stdout. From the "Replacing shell pipeline" section of the subprocess docs, you do..

    p1 = Popen(["dmesg"], stdout=PIPE)
    p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
    

    ..whereas you were doing the equivalent of stdin=p1.

    The tr1 = (in the above rewritten code) line would become..

    tr1 = Popen(['tr', '-s', r"'\040'"], stdin=blame.stdout, stdout=PIPE)
    

  • You do not need to escape commands/arguments with subprocess, as subprocess does not run the command in any shell (unless you specify shell=True). See the Securitysection of the subprocess docs.

    Instead of..

    proc.append(Popen(['svn', 'blame', shellquote(filename)], stdout=PIPE))
    

    ..you can safely do..

    Popen(['svn', 'blame', filename], stdout=PIPE)
    

  • As S.Lott suggested, don't use subprocess to do text-manipulations easier done in Python (the tr/cut commands). For one, tr/cut etc aren't hugely portable (different versions have different arguments), also they are quite hard to read (I've no idea what the tr's and cut are doing)

    If I were to rewrite the command, I would probably do something like..

    def get_blame(filename): 
        blame = Popen(['svn', 'blame', filename], stdout=PIPE)
        output = blame.communicate()[0] # preferred to blame.stdout.read()
        # process commands output:
        ret = []
        for line in output.split("\n"):
            split_line = line.strip().split(" ")
            if len(split_line) > 2:
                rev = split_line[0]
                author = split_line[1]
                line = " ".join(split_line[2:])
    
    
            ret.append({'rev':rev, 'author':author, 'line':line})
    
    
    return ret
    
    
    

• Like S.Lott said, processing the text in Python is better.

  But if you want to use the cmdline utilities, you can keep it readable by using shell=True:

  cmdline = r"svn blame %s | tr -s '\040' | tr '\040' ';' | cut -d \; -f 3" % shellquote(filename)
  return Popen(cmdline, shell=True, stdout=PIPE).communicate()[0]