Python subprocess with real-time input and multiple consoles

The main issue

I have a main command line Python script, which is supposed to open an additional console for log output only. For this I intend to redirect the log output, which would be printed on the main script's console, to the stdin of the second console, which I start as a subprocess. (I use subprocess, because I didn't find any other way to open a second console.)

The problem is, that it seems that I'm able to send to the stdin of this second console - however, nothing gets printed on this second console.

Following is the code I used for experimenting (with Python 3.4 on PyDev under Windows 10). The function writing(input, pipe, process) contains the part, where the generated string is copied to the as pipe passed stdin, of the via subprocess opened console. The function writing(...) is run via the class writetest(Thread). (I left some code, which I commented out.)

import os
import sys
import io
import time
import threading
from cmd import Cmd
from queue import Queue
from subprocess import Popen, PIPE, CREATE_NEW_CONSOLE


REPETITIONS = 3


class generatetest(threading.Thread):

    def __init__(self, queue):
        self.output = queue
        threading.Thread.__init__(self)

    def run(self):
        print('run generatetest')
        generating(REPETITIONS, self.output)
        print('generatetest done')

    def getout(self):
        return self.output


class writetest(threading.Thread):

    def __init__(self, input=None, pipe=None, process=None):
        if (input == None):        # just in case
            self.input = Queue()
        else:
            self.input = input

        if (pipe == None):        # just in case
            self.pipe = PIPE
        else:
            self.pipe = pipe

        if (process == None):        # just in case
            self.process = subprocess.Popen('C:\Windows\System32\cmd.exe', universal_newlines=True, creationflags=CREATE_NEW_CONSOLE)
        else:
            self.process = proc

        threading.Thread.__init__(self)

    def run(self):
        print('run writetest')
        writing(self.input, self.pipe, self.process)
        print('writetest done')


def generating(maxint, outline):
    print('def generating')
    for i in range(maxint):
        time.sleep(1)
        outline.put_nowait(i)


def writing(input, pipe, process):
    print('def writing')
    while(True):
        try:
            print('try')
            string = str(input.get(True, REPETITIONS)) + "\n"
            pipe = io.StringIO(string)
            pipe.flush()
            time.sleep(1)
            # print(pipe.readline())
        except:
            print('except')
            break
        finally:
            print('finally')
            pass


data_queue = Queue()
data_pipe = sys.stdin
# printer = sys.stdout
# data_pipe = os.pipe()[1]

exe = 'C:\Python34\python.exe'
# exe = 'C:\Windows\System32\cmd.exe'
arg = 'C:\\Users\\Public\\Documents\\test\\test-cmd.py'
arguments = [exe, arg]
# proc = Popen(arguments, universal_newlines=True, creationflags=CREATE_NEW_CONSOLE)
proc = Popen(arguments, stdin=data_pipe, stdout=PIPE, stderr=PIPE,
             universal_newlines=True, creationflags=CREATE_NEW_CONSOLE)

thread1 = generatetest(data_queue)
thread2 = writetest(data_queue, data_pipe, proc)

# time.sleep(5)

thread1.start()
thread2.start()

thread1.join(REPETITIONS * REPETITIONS)
thread2.join(REPETITIONS * REPETITIONS)

# data_queue.join()
# receiver = proc.communicate(stdin, 5)
# print('OUT:' + receiver[0])
# print('ERR:' + receiver[1])

print("1st part finished")

---

A slightly different approach

The following additional code snippet works in regard to extracting the stdout from the subprocess. However, the previously sent stdin still isn't print on the second console.

proc2 = Popen(['C:\Python34\python.exe', '-i'],
              stdin=PIPE,
              stdout=PIPE,
              stderr=PIPE,
              creationflags=CREATE_NEW_CONSOLE)
proc2.stdin.write(b'2+2\n')
proc2.stdin.flush()
print(proc2.stdout.readline())
proc2.stdin.write(b'len("foobar")\n')
proc2.stdin.flush()
print(proc2.stdout.readline())
time.sleep(1)
proc2.stdin.close()
proc2.terminate()
proc2.wait(timeout=0.2)

print("Exiting Main Thread")

---

More info

As soon as I use one of the paramaters stdin=data_pipe, stdout=PIPE, stderr=PIPE for starting the subprocess, the resulting second console isn't active and does't accept keyboard input.

The subprocess method communicate() can't be used for this as it waits for the process to end.

---

More code

Finally the code for the file, which is for the second console.

C:\Users\Public\Documents\test\test-cmd.py

from cmd import Cmd
from time import sleep
from datetime import datetime

INTRO = 'command line'
PROMPT = '> '


class CommandLine(Cmd):
    """Custom console"""

    def __init__(self, intro=INTRO, prompt=PROMPT):
        Cmd.__init__(self)
        self.intro = intro
        self.prompt = prompt
        self.doc_header = intro
        self.running = False

    def do_dummy(self, args):
        """Runs a dummy method."""
        print("Do the dummy.")
        self.running = True
        while(self.running == True):
            print(datetime.now())
            sleep(5)

    def do_stop(self, args):
        """Stops the dummy method."""
        print("Stop the dummy, if you can.")
        self.running = False

    def do_exit(self, args):
        """Exits this console."""
        print("Do console exit.")
        exit()

if __name__ == '__main__':
    cl = CommandLine()
    cl.prompt = PROMPT
    cl.cmdloop(INTRO)

---

Thoughts

So far I'm even not certain if the Windows command line interface offers the capability to accept other input than the one from the keyboard. Though, with it having some sort of passive mode, I expect it.

Why is this not working?

from Python subprocess with real-time input and multiple consoles