[vpsFree.cz: community-list] {Disarmed} {Disarmed} Re: php pro nginx high load

Jaroslav Skřivan skrivy at skrivy.net
Wed Dec 2 13:40:04 CET 2015 

To si uplne nemyslim. Nginx se pripoji na port a preda pozadavek 
procesu, ktery zrovna ceka na accept (resp. fcgi_accept_request), pokud 
zadny takovy neni, tak ceka v backlogu. Kdyz FPM ukoncuje proces, tak 
ten proces listen socket uzavre a nginx se k nemu nedoboucha.

Jarda

------ Original Message ------
From: "Tomáš Jirman" <tjirman at gmail.com>
To: "vpsFree.cz Community list" <community-list at lists.vpsfree.cz>
Cc: "veros kaplan" <veros.kaplan at koren.cz>
Sent: 12/2/2015 1:18:58 PM
Subject: Re: [vpsFree.cz: community-list] {Disarmed} {Disarmed} Re: php 
pro nginx high load

>Tohle jsem řešil nedávno taky. Uprav počet pm.max_requests na cca 10x 
>až 100x vyšší číslo, nebo ho nastav na nulu pokud víš, že ti v aplikaci 
>neleakuje paměť. Fpm se chová tak, že po splnění limitu requestů ten 
>child zabije, ale nginx se na něj zkusí stejně připojit a skončí to 
>touhle chybou.
>
>Tomáš
>
>st 2. 12. 2015 v 11:21 odesílatel <skrivy at skrivy.net> napsal:
>>To je jako zavolat doktorovi, ze ti je spatne. :) Na to neni obecna 
>>odpoved. Vsechny aplikace jsou jine a potrebuji trochu neco jineho.
>>
>>
>>
>>Jak uz bylo zmineno - opcode cache, vyhnout se zamkum na memcache, 
>>atd. S temi procesy bych byl opatrny - php casto ceka na vnejsi 
>>zdroje, takze se jich tam da spustit vic.
>>
>>
>>
>>Monitoring delam by default scenarem v zabbixu, aby byla nejaka 
>>dlouhodoba data + samozrejne nginx server status a php status.
>>
>>
>>
>>U problemovych webu sbirame i data z aplikacni urovne - tzn. jak 
>>dlouho trvaji volani do ..., sql dotazy, http kody a vsechno to sypu 
>>do grafu, abych videl dlouhodobe zmeny.
>>
>>
>>
>>--
>>
>>
>>
>>Sent from my Nokia N9
>>
>>
>>
>>
>>On 02.12.2015 11:03 Zlatko Fedor wrote:
>>
>>Velmi rad pocujem ze niekomu to funguje spravne.
>>Vies prosim napisat viac informacii pripadne prilozit svoj config?
>>Rad si aspon spatne precitam co som robil zle.
>>
>>Nasledujucu otazku si prosim neber osobne:
>>Monitoruju tvoje weby uplne kazdy vypadok a chybu alebo sa len 
>>spoliehas na to ze to ide ked napises stranku do browsera pripadne na 
>>pingdom ktoremu vela veci ujde.
>>
>>Dakujem za odpoved
>>
>>
>>
>>>On 02 Dec 2015, at 10:36, skrivy at skrivy.net wrote:
>>>
>>>Proti tomu se musim ohradit.
>>>
>>>Hostuju weby z alexa 15 000 a se spravnou konfiguraci a slusne 
>>>napsanou aplikaci s tim neni jedinej problem.
>>>
>>>
>>>--
>>>
>>>
>>>Sent from my Nokia N9
>>>
>>>
>>>
>>>On 02.12.2015 10:23 Zlatko Fedor wrote:
>>>
>>>Ahoj.
>>>
>>>Pred par mesiacmi som mal podobny problem.
>>>Skusil som naozaj vsetko. Milion nastaveni no stale to robilo to iste 
>>>s mensim alebo vecsim opakovanim.
>>>Skusal som unix sockery, tcp sokety, rozne cache, dynamic/static 
>>>childs proste setko co som nasiel. Precital som celu dokumentaciu 
>>>takze som presne vedel co robim.
>>>Po pol roku som to vzdal a prepisal vsetko do node.js. Podla mojho 
>>>nazoru je PHP-FPM uplne zabagovane.
>>>Predtym som pouzival apache s php a nemal som jediny problem. Ale asi 
>>>ako ty aj ja som narazil vtedy na jeho limity.
>>>
>>>Viem ze som ti asi nepomohol len ta chcem pripravit aj na moznost ze 
>>>PHP FPM nieje kvalitny softver.
>>>
>>>S pozdravom
>>>Zlatko Fedor
>>>
>>>>On 02 Dec 2015, at 09:56, Branislav Viest <info at branoviest.com> 
>>>>wrote:
>>>>
>>>>tohle je vytezek z php-fpm logu
>>>>
>>>>[02-Dec-2015 09:54:39.867720] WARNING: pid 5043, 
>>>>fpm_request_check_timed_out(), line 269: [pool web1] child 24945, 
>>>>script 'xxxwww//index.php' (request: "GET /index.php") executing too 
>>>>slow (5.799737 sec), logging # to je jasny
>>>>[02-Dec-2015 09:54:39.867981] DEBUG: pid 5043, fpm_got_signal(), 
>>>>line 76: received SIGCHLD
>>>>[02-Dec-2015 09:54:39.868499] NOTICE: pid 5043, fpm_children_bury(), 
>>>>line 227: child 24945 stopped for tracing
>>>>[02-Dec-2015 09:54:39.868508] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 144: about to trace 24945
>>>>[02-Dec-2015 09:54:39.868731] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 172: finished trace of 24945
>>>>[02-Dec-2015 09:54:39.869145] NOTICE: pid 5043, fpm_children_bury(), 
>>>>line 227: child 28359 stopped for tracing
>>>>[02-Dec-2015 09:54:39.869161] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 144: about to trace 28359
>>>>[02-Dec-2015 09:54:39.869415] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 172: finished trace of 28359
>>>>[02-Dec-2015 09:54:39.869893] NOTICE: pid 5043, fpm_children_bury(), 
>>>>line 227: child 33445 stopped for tracing
>>>>[02-Dec-2015 09:54:39.869904] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 144: about to trace 33445
>>>>[02-Dec-2015 09:54:39.870101] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 172: finished trace of 33445
>>>>[02-Dec-2015 09:54:39.870602] NOTICE: pid 5043, fpm_children_bury(), 
>>>>line 227: child 36581 stopped for tracing
>>>>[02-Dec-2015 09:54:39.870615] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 144: about to trace 36581
>>>>[02-Dec-2015 09:54:39.870884] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 172: finished trace of 36581
>>>>[02-Dec-2015 09:54:39.871187] NOTICE: pid 5043, fpm_children_bury(), 
>>>>line 227: child 37001 stopped for tracing
>>>>[02-Dec-2015 09:54:39.871195] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 144: about to trace 37001
>>>>[02-Dec-2015 09:54:39.871460] NOTICE: pid 5043, fpm_php_trace(), 
>>>>line 172: finished trace of 37001
>>>>[02-Dec-2015 09:54:39.872162] DEBUG: pid 5043, fpm_got_signal(), 
>>>>line 76: received SIGCHLD
>>>>[02-Dec-2015 09:54:39.872582] DEBUG: pid 5043, fpm_got_signal(), 
>>>>line 76: received SIGCHLD
>>>>[02-Dec-2015 09:54:39.872950] DEBUG: pid 5043, fpm_got_signal(), 
>>>>line 76: received SIGCHLD
>>>>[02-Dec-2015 09:54:39.873295] DEBUG: pid 5043, fpm_got_signal(), 
>>>>line 76: received SIGCHLD
>>>>[02-Dec-2015 09:54:39.873616] DEBUG: pid 5043, fpm_event_loop(), 
>>>>line 424: event module triggered 1 events
>>>>
>>>>S pozdravem
>>>>Branislav Viest
>>>>
>>>>
>>>>--------------------------------------------------------------------------------
>>>>Od: "Věroslav Kaplan" <veros.kaplan at gmail.com>
>>>>Komu: "vpsFree.cz Community list" <community-list at lists.vpsfree.cz>
>>>>Odoslané: streda, 2. december 2015 9:48:22
>>>>Predmet: {Disarmed} [vpsFree.cz: community-list] {Disarmed} Re: php 
>>>>pro nginx high load
>>>>
>>>>Ahoj,
>>>>
>>>>Tohle vypadá, že PHP-FPM z nějakého důvodu zavře to fcgi spojení. 
>>>>Dokážeš vytáhnout nějaké podezřelé logy z toho PHP-FPM a zkusit 
>>>>uhodnout, proč to dělá ?
>>>>
>>>>--Věroš Kaplan
>>>>http://veroskaplan.cz/
>>>>Školíme Ansible: http://ansible.cz/
>>>>
>>>>2015-12-02 8:25 GMT+01:00 Branislav Viest <info at branoviest.com>:
>>>>>Ahoj,
>>>>>
>>>>>mám tady jeden server kde je cca 35k lidí za vteřinu. Běží tu nginx 
>>>>>+ php fpm a mysql db. V nginxu jsem rozjel microcache, což hodně 
>>>>>ulevilo zátěži serveru. Nicméně, mám problém s PHPkem. FPM běží 
>>>>>přes TCP (zkoušel jsem i unix socket nicméně bylo to pomalější a 
>>>>>docházelo ke stejnému problému). Dle error logu dochází k těmto 
>>>>>chybám:
>>>>>
>>>>>2015/12/01 23:26:56 [error] 27043#0: *122476 recv() failed (104: 
>>>>>Connection reset by peer) while reading response header from 
>>>>>upstream, client: 188.114.98.51, server: xxx request: "GET 
>>>>>/xxx/xxxx/xxx HTTP/1.1", upstream: "fastcgi://MailScanner has 
>>>>>detected a possible fraud attempt from "127.0.0.1:9000" claiming to 
>>>>>beMailScanner warning: numerical links are often malicious: 
>>>>>127.0.0.1:9000", host: "xxxx", referrer: "MailScanner has detected 
>>>>>a possible fraud attempt from "xxxxx" claiming to be http://xxxxx"
>>>>>2015/12/01 23:26:56 [error] 27032#0: *124368 recv() failed (104: 
>>>>>Connection reset by peer) while reading response header from 
>>>>>upstream, client: 188.114.99.50, server: xxxxxx, request: "GET 
>>>>>/xxxxx/xxxxx HTTP/1.1", upstream: "fastcgi://MailScanner has 
>>>>>detected a possible fraud attempt from "127.0.0.1:9000" claiming to 
>>>>>beMailScanner warning: numerical links are often malicious: 
>>>>>127.0.0.1:9000", host: "xxxxx", referrer: "xxxxx"
>>>>>
>>>>>a sype to tam docela drsně. Jak vypnu kešování v nginxu load 
>>>>>vyskočí, třeba i na 500, těch chyb PHPka je méně ale jsou tam 
>>>>>pořád.
>>>>>
>>>>>Konfigurace FPM pro pool toho webu je:
>>>>>
>>>>>[web1]
>>>>>
>>>>>listen = MailScanner has detected a possible fraud attempt from 
>>>>>"127.0.0.1:9000" claiming to beMailScanner warning: numerical links 
>>>>>are often malicious: 127.0.0.1:9000
>>>>>listen.allowed_clients = 127.0.0.1
>>>>>listen.owner = web1
>>>>>listen.group = client0
>>>>>listen.mode = 0660
>>>>>listen.backlog = 65536
>>>>>
>>>>>user = web1
>>>>>group = client0
>>>>>
>>>>>request_slowlog_timeout = 5s
>>>>>slowlog = /var/log/php-fpm/slowlog-web1.log
>>>>>
>>>>>pm = dynamic
>>>>>pm.max_children = 7000
>>>>>pm.start_servers = 4000
>>>>>pm.min_spare_servers = 2000
>>>>>pm.max_spare_servers = 4000
>>>>>pm.max_requests = 100
>>>>>
>>>>>request_terminate_timeout = 60s
>>>>>rlimit_files = 500000
>>>>>rlimit_core = unlimited
>>>>>catch_workers_output = yes
>>>>>
>>>>>pm.status_path = /php-status
>>>>>
>>>>>chdir = /
>>>>>
>>>>>+ openbasedir, sessions save path. atp.
>>>>>
>>>>>S těma hodnotama jsem se zkoušel různě hrát, nicméně to k vyřešení 
>>>>>tohoto problému nepomohlo. Zkoušel jsem upravit i nějaké parametry 
>>>>>jádra (local port range, tcp ack timeouty atp.) ale výsledek 
>>>>>stejný. Dělá to i u unix socketu, což je mi právě zvláštní. Nginx 
>>>>>konfigurace je:
>>>>>user www-data;
>>>>>worker_processes 80;
>>>>>pid /run/nginx.pid;
>>>>># set open fd limit to 50000
>>>>>worker_rlimit_nofile 100000;
>>>>>
>>>>>events {
>>>>>worker_connections 1024;
>>>>>multi_accept on;
>>>>>use epoll;
>>>>>}
>>>>>
>>>>>ve vhostu nic zvlastniho, jenom pro php:
>>>>>
>>>>>fastcgi_split_path_info ^(.+\.php)(/.+)$;
>>>>>fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
>>>>>include /etc/nginx/fastcgi_params;
>>>>>fastcgi_pass MailScanner has detected a possible fraud attempt from 
>>>>>"127.0.0.1:9000" claiming to beMailScanner warning: numerical links 
>>>>>are often malicious: 127.0.0.1:9000;
>>>>>fastcgi_index index.php;
>>>>>fastcgi_intercept_errors on;
>>>>>fastcgi_read_timeout 4m;
>>>>>
>>>>>Dnes bude k tomuto server nový, pro rozdělení zátěže, ale jako fakt 
>>>>>nevím jestli to má smysl ve stejné konfiguraci, kvůli těm php 
>>>>>chybám. Nenapadá Vás kluci, co s tím? Nebo případně návrh řešení, 
>>>>>kterým bych nahradil tohle, při dvou serverech ? Přemýšlím nad 
>>>>>MySQL M - M replikací, v DNS round robin a GlusterFS pro data. Ale 
>>>>>to phpko mi nedá spát. A dnes večer to už další nápor asi nedá.
>>>>>
>>>>>Díky za rady a konzultace.
>>>>>
>>>>>S pozdravem
>>>>>Branislav Viest
>>>>>
>>>>>_______________________________________________
>>>>>Community-list mailing list
>>>>>Community-list at lists.vpsfree.cz
>>>>>http://lists.vpsfree.cz/listinfo/community-list
>>>>>
>>>>
>>>>
>>>>
>>>>--
>>>>--V.Kaplan
>>>>
>>>>_______________________________________________
>>>>Community-list mailing list
>>>>Community-list at lists.vpsfree.cz
>>>>http://lists.vpsfree.cz/listinfo/community-list
>>>>_______________________________________________
>>>>Community-list mailing list
>>>>Community-list at lists.vpsfree.cz
>>>>http://lists.vpsfree.cz/listinfo/community-list
>>>
>>>
>>>_______________________________________________
>>>Community-list mailing list
>>>Community-list at lists.vpsfree.cz
>>>http://lists.vpsfree.cz/listinfo/community-list
>>
>>
>>_______________________________________________
>>Community-list mailing list
>>Community-list at lists.vpsfree.cz
>>http://lists.vpsfree.cz/listinfo/community-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.vpsfree.cz/pipermail/community-list/attachments/20151202/a2e24f48/attachment-0002.html>

More information about the Community-list mailing list