python - How to increase connection pool size for Twisted? -

-

I am using twisted 8.1.0 as a socket server engine. Reactor - Apol database server is MySQL 5.0.67. OS - Ubuntu Linux 8.10 32-bit

In the /etc/mysql/my.cnf : 

  max_connections = 1000  < / Pre> 
 In source code: 
 
  adbapi.ConnectionPool ("MySQLdb", ..., use_unicode = true, charset = 'utf8', cp_min = 3, cp_max = 700, Cp_noisy = false]  
 
 But in reality, I can only see 200 (or less) open connections ( show processist ) when applications are under heavy load This is not enough for my app: (
 
 As I have seen that this is the limit of thread pools. Any ideas?

As you suspect, this is probably a threading problem. Thread pool determines an upper limit for the number of threads in the thread, however, your process is likely to run from this limit Out of less memory, around 200 threads in your case, because each thread has its own stack, the total memory used by your process hits the system limit and no thread is created. can.

Before you run your program, you can check it by adjusting the stack size setting (I am using bash ), i.e. 

 $ Ulimit - A core file size (block, -C) 0 data sized size (kbytes, -d) Unlimited maximum good (-e) 0 file size (block, -f) Unlimited pending signals (-i) 32750 Maximum lock memory (kbytes, -l) 32 Maximum memory size (KB tes, -m) Unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX messaging (Bytes, -c) 819200 Maximum RT priority (-r) 0 stack size (kilobyte, -s) 10240 CPU time (seconds, -T) Unlimited maximum user processes (-U) 32750 virtual memory (kbts, -v) Unlimited file lock (-x) unlimited

You can see that the default stack size is 10240K on my machine and I have found that I can make around 300 threads with this setting. Adjusting the stack size to 1024K (using the ulimit -s 1024 ) I can create about 3,000 threads.

You can get some information about the thread construction limits on the system by using this script:

Import from thread import_in_thread time import sleeping deep sleeper ): Try: While leaving 1: sleep (10000): If running: def test (): global running n = 0 is going on = try true: while 1: start_new_thread (sleeper, ()) exception Except n = = 1, e: current = wrong print 'Exception:', and print 'largest number of threads:' , N if __name__ == '

this resolves your problem, it will depend on the memory requirements of the connectionflat threads.


Comments

Post a Comment

Popular posts from this blog

c# - ListView onScroll event -

-
I am programming a simple C # application, and I need a Scallow event on the list view. So I inherited the original ListView class created in ListviewEx witch. I found out how to locate the scroll message from WinAPI and I modified the WndProc method. Now I have this WndProc: protected override zero WndProc (Ref message message) {base.WndProc (Ref I); If (m.msg == WM_VSCROLL) {onScroll (This, New EventArgs ()); }} But the problem is, I do not know how to know about scrolling information. This data should be in wParam, but there is a LOWORD macro ++ like in C in C # and I switch to find the criteria like SB_ below, SB_ENDSCROLL, SB_PAGEUP etc. there any way Need to change the LOWORD macro from C #? Or any other way to find the necessary parameters about scrolling? You can define the WParam constants as the following: Private const int WM_HSCROLL = 0x114; Private Constant WM_VSCROLL = 0x115; Private contact int SBHOZZ = 0; Private Consultant SB_VERT = 1; Private Con...
Read more

PHP - get image from byte array -

-
I am trying to create a back-end for my mobile application. I am sending the content of an image in the body of the HTTP request (output stream) as a byte array, I want to read this stream of bytes in PHP scripts and take back an image. Can anyone tell me how can I do this? Thank you. You can obtain the request body $ body = Reading from file_get_contents ('php: // input'); What you do with that data Depending on you, you can write the data in a file as you mentioned that this is image data, you can drop the data into the object can do. Another option is to use and load the image.
Read more

Linux Terminal Problem with Non-Canonical Terminal I/O app -

-
I have a small app written in C to run on Linux. Part of the app accepts user-input from keyboard , And it uses non-canonical terminal mode so that it can respond to each keystroke. The section of the code that accepts input is a simple task that is called repeatedly in the loop: char get_input () {char C = 0; Int res = reading (input_tamille, & amp; c, 1); If (RSS == 0) 0 returns; If (res == -1) {/ * snip error handling * /} return c; } It reads a letter from the terminal if there is no input within a given time limit, (c_cc [VTIME] value in the vocabulary structure), read () 0 And get_input () is called again It all works very well, I have recently discovered that if you run this app in a terminal window, and then the app Close the terminal window without ending, the app will not exit Received, but CPU intensive launches into an infinite loop, where the read () returns 0 without waiting continued. So how can I get out of the apple if it runs from the terminal win...
Read more