At first, i thought that max open file limit was the problem, to verify i did a manual checking as seen below.
438.. ôO'
In [1]: count = 0
In [2]: from threading import Thread
In [3]: def blah():
...: pass
...:
In [4]: while 1:
...: Thread(target=blah).start()
...: count += 1
...:
...:
---------------------------------------------------------------------------
error Traceback (most recent call last)
/root/ in ()
/usr/local/lib/python2.5/threading.pyc in start(self)
438 _limbo[self] = self
439 _active_limbo_lock.release()
--> 440 _start_new_thread(self.__bootstrap, ())
441 self.__started = True
442 _sleep(0.000001) # 1 usec, to let the thread run (Solaris hack)
error: can't start new thread
In [5]: print count
438
i was expecting 1024 or so(mostly more than 1024 if running as root) but 438 was way lower than that. It seemed that another problem was going on.
root@s1 [~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 65536
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 65536
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
stack_size(10mb) seems to be the culprit here. stack_size is basically the maximum memory size a single process can allocate as stack.
10Mbs is much more than needed for my threads, so retrying the manual checking succeeded as expected.
In [1]: import threading as t
In [2]: def blah():
...: pass
...:
In [3]: t.stack_size(128*1024) # 128K would be enough in the real code, so trying it here
Out[3]: 0
In [4]: count = 0
In [5]: while 1:
...: t.Thread(target=blah).start()
...: count +=1
...:
...:
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
.........
KeyboardInterrupt: # bored waiting :o/
In [6]: count
Out[6]: 68348 <------ Yays! Now i'm happily enjoying my gazillions of semi-concurrent threads.
0 comments:
Post a Comment