<HTML><HEAD>
<STYLE id=eMClientCss>blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc }
blockquote.cite2 {margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc; margin-top: 3px; padding-top: 0px; }
.plain pre, .plain tt { font-family: monospace; font-size: 100%; font-weight: normal; font-style: normal; white-space: pre-wrap; }
a img { border: 0px; }body {font-family: Tahoma;font-size: 12pt;}
.plain pre, .plain tt {font-family: Tahoma;font-size: 12pt;}
</STYLE>
<STYLE>#x0a98f24b0253481583377c9a18a39dc8 BLOCKQUOTE.cite2 {
PADDING-TOP: 0px; PADDING-LEFT: 10px; MARGIN-LEFT: 5px; BORDER-LEFT: #cccccc 1px solid; MARGIN-TOP: 3px; PADDING-RIGHT: 0px; MARGIN-RIGHT: 0px
}
#x0a98f24b0253481583377c9a18a39dc8 .plain PRE {
FONT-SIZE: 100%; FONT-FAMILY: monospace; FONT-WEIGHT: normal; FONT-STYLE: normal
}
.plain TT {
FONT-SIZE: 100%; FONT-FAMILY: monospace; FONT-WEIGHT: normal; FONT-STYLE: normal
}
#x0a98f24b0253481583377c9a18a39dc8 A IMG {
BORDER-TOP: 0px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px
}
#x0a98f24b0253481583377c9a18a39dc8 {
FONT-SIZE: 12pt; FONT-FAMILY: Tahoma
}
#x0a98f24b0253481583377c9a18a39dc8 .plain PRE {
FONT-SIZE: 12pt; FONT-FAMILY: Tahoma
}
.plain TT {
FONT-SIZE: 12pt; FONT-FAMILY: Tahoma
}
</STYLE>
</HEAD>
<BODY style="FONT-SIZE: 12pt; FONT-FAMILY: Tahoma" scroll=auto class>
<DIV><SPAN>To si uplne nemyslim. Nginx se pripoji na port a preda pozadavek procesu, ktery zrovna ceka na accept (resp. fcgi_accept_request), pokud zadny takovy neni, tak ceka v backlogu. Kdyz FPM ukoncuje proces, tak ten proces listen socket uzavre a nginx se k nemu nedoboucha.</SPAN></DIV>
<DIV><SPAN></SPAN> </DIV>
<DIV><SPAN>Jarda</SPAN></DIV>
<DIV> </DIV>
<DIV>------ Original Message ------</DIV>
<DIV>From: "Tomáš Jirman" <<A href="mailto:tjirman@gmail.com">tjirman@gmail.com</A>></DIV>
<DIV>To: "vpsFree.cz Community list" <<A href="mailto:community-list@lists.vpsfree.cz">community-list@lists.vpsfree.cz</A>></DIV>
<DIV>Cc: "veros kaplan" <<A href="mailto:veros.kaplan@koren.cz">veros.kaplan@koren.cz</A>></DIV>
<DIV>Sent: 12/2/2015 1:18:58 PM</DIV>
<DIV>Subject: Re: [vpsFree.cz: community-list] {Disarmed} {Disarmed} Re: php pro nginx high load</DIV>
<DIV> </DIV>
<DIV id=x0d2160c3c40d47d99de3e4c333c5bdf4>
<BLOCKQUOTE class=cite2 cite=CAJSe15xjxUN0F8NEyPq1k0n_ga+abQ+7fEPbGtrcorga9h6pRg@mail.gmail.com type="cite">
<DIV dir=ltr>Tohle jsem řešil nedávno taky. Uprav počet pm.max_requests na cca 10x až 100x vyšší číslo, nebo ho nastav na nulu pokud víš, že ti v aplikaci neleakuje paměť. Fpm se chová tak, že po splnění limitu requestů ten child zabije, ale nginx se na něj zkusí stejně připojit a skončí to touhle chybou.
<DIV><BR></DIV>
<DIV>Tomáš</DIV><BR>
<DIV class=gmail_quote>
<DIV dir=ltr>st 2. 12. 2015 v 11:21 odesílatel <<A href="mailto:skrivy@skrivy.net">skrivy@skrivy.net</A>> napsal:<BR></DIV></DIV>
<DIV class=gmail_quote>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV style="WORD-WRAP: break-word">
<DIV>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">To je jako zavolat doktorovi, ze ti je spatne. :) Na to neni obecna odpoved. Vsechny aplikace jsou jine a potrebuji trochu neco jineho.</SPAN></P>
<P style="FONT-FAMILY: 'NokiaPureTextLight'; MARGIN: 0px; TEXT-INDENT: 0px"><BR></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Jak uz bylo zmineno - opcode cache, vyhnout se zamkum na memcache, atd. S temi procesy bych byl opatrny - php casto ceka na vnejsi zdroje, takze se jich tam da spustit vic.</SPAN></P>
<P style="FONT-FAMILY: 'NokiaPureTextLight'; MARGIN: 0px; TEXT-INDENT: 0px"><BR></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Monitoring delam by default scenarem v zabbixu, aby byla nejaka dlouhodoba data + samozrejne nginx server status a php status.</SPAN></P>
<P style="FONT-FAMILY: 'NokiaPureTextLight'; MARGIN: 0px; TEXT-INDENT: 0px"><BR></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">U problemovych webu sbirame i data z aplikacni urovne - tzn. jak dlouho trvaji volani do ..., sql dotazy, http kody a vsechno to sypu do grafu, abych videl dlouhodobe zmeny.</SPAN></P></DIV></DIV>
<DIV style="WORD-WRAP: break-word">
<DIV>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">--</SPAN></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Sent from my Nokia N9</SPAN></P>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P></DIV></DIV>
<DIV style="WORD-WRAP: break-word"><BR>
<DIV>
<P>On 02.12.2015 11:03 Zlatko Fedor wrote:<BR></P></DIV>
<DIV>
<DIV>Velmi rad pocujem ze niekomu to funguje spravne. </DIV>
<DIV>Vies prosim napisat viac informacii pripadne prilozit svoj config?</DIV>
<DIV>Rad si aspon spatne precitam co som robil zle.</DIV>
<DIV><BR></DIV>
<DIV>Nasledujucu otazku si prosim neber osobne:</DIV>
<DIV>Monitoruju tvoje weby uplne kazdy vypadok a chybu alebo sa len spoliehas na to ze to ide ked napises stranku do browsera pripadne na pingdom ktoremu vela veci ujde.</DIV>
<DIV><BR></DIV>
<DIV>Dakujem za odpoved</DIV>
<DIV><BR></DIV>
<DIV><BR></DIV><BR>
<DIV>
<BLOCKQUOTE class=cite type="cite">
<DIV>On 02 Dec 2015, at 10:36, <A href="mailto:skrivy@skrivy.net">skrivy@skrivy.net</A> wrote:</DIV><BR>
<DIV>
<DIV style="WORD-WRAP: break-word">
<DIV>
<DIV style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Proti tomu se musim ohradit.</SPAN></DIV>
<DIV style="FONT-FAMILY: NokiaPureTextLight; MARGIN: 0px; TEXT-INDENT: 0px"><BR></DIV>
<DIV style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Hostuju weby z alexa 15 000 a se spravnou konfiguraci a slusne napsanou aplikaci s tim neni jedinej problem.</SPAN></DIV>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P>
<DIV style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">--</SPAN></DIV>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P>
<DIV style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'">Sent from my Nokia N9</SPAN></DIV>
<P style="MARGIN: 0px; TEXT-INDENT: 0px"><SPAN style="FONT-FAMILY: 'NokiaPureTextLight'"> </SPAN></P></DIV><BR>
<DIV>
<P>On 02.12.2015 10:23 Zlatko Fedor wrote:<BR></P></DIV>
<DIV>Ahoj.
<DIV><BR></DIV>
<DIV>Pred par mesiacmi som mal podobny problem.</DIV>
<DIV>Skusil som naozaj vsetko. Milion nastaveni no stale to robilo to iste s mensim alebo vecsim opakovanim.</DIV>
<DIV>Skusal som unix sockery, tcp sokety, rozne cache, dynamic/static childs proste setko co som nasiel. Precital som celu dokumentaciu takze som presne vedel co robim.</DIV>
<DIV>Po pol roku som to vzdal a prepisal vsetko do node.js. Podla mojho nazoru je PHP-FPM uplne zabagovane.</DIV>
<DIV>Predtym som pouzival apache s php a nemal som jediny problem. Ale asi ako ty aj ja som narazil vtedy na jeho limity.</DIV>
<DIV><BR></DIV>
<DIV>Viem ze som ti asi nepomohol len ta chcem pripravit aj na moznost ze PHP FPM nieje kvalitny softver.</DIV>
<DIV><BR></DIV>
<DIV>S pozdravom </DIV>
<DIV>Zlatko Fedor</DIV>
<DIV><BR>
<DIV>
<BLOCKQUOTE class=cite type="cite">
<DIV>On 02 Dec 2015, at 09:56, Branislav Viest <<A href="mailto:info@branoviest.com">info@branoviest.com</A>> wrote:</DIV><BR>
<DIV>
<DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: arial,helvetica,sans-serif">
<DIV>tohle je vytezek z php-fpm logu<BR></DIV>
<DIV><BR></DIV>
<DIV>[02-Dec-2015 09:54:39.867720] WARNING: pid 5043, fpm_request_check_timed_out(), line 269: [pool web1] child 24945, script 'xxxwww//index.php' (request: "GET /index.php") executing too slow (5.799737 sec), logging # to je jasny<BR>[02-Dec-2015 09:54:39.867981] DEBUG: pid 5043, fpm_got_signal(), line 76: received SIGCHLD<BR>[02-Dec-2015 09:54:39.868499] NOTICE: pid 5043, fpm_children_bury(), line 227: child 24945 stopped for tracing<BR>[02-Dec-2015 09:54:39.868508] NOTICE: pid 5043, fpm_php_trace(), line 144: about to trace 24945<BR>[02-Dec-2015 09:54:39.868731] NOTICE: pid 5043, fpm_php_trace(), line 172: finished trace of 24945<BR>[02-Dec-2015 09:54:39.869145] NOTICE: pid 5043, fpm_children_bury(), line 227: child 28359 stopped for tracing<BR>[02-Dec-2015 09:54:39.869161] NOTICE: pid 5043, fpm_php_trace(), line 144: about to trace 28359<BR>[02-Dec-2015 09:54:39.869415] NOTICE: pid 5043, fpm_php_trace(), line 172: finished trace of 28359<BR>[02-Dec-2015 09:54:39.869893] NOTICE: pid 5043, fpm_children_bury(), line 227: child 33445 stopped for tracing<BR>[02-Dec-2015 09:54:39.869904] NOTICE: pid 5043, fpm_php_trace(), line 144: about to trace 33445<BR>[02-Dec-2015 09:54:39.870101] NOTICE: pid 5043, fpm_php_trace(), line 172: finished trace of 33445<BR>[02-Dec-2015 09:54:39.870602] NOTICE: pid 5043, fpm_children_bury(), line 227: child 36581 stopped for tracing<BR>[02-Dec-2015 09:54:39.870615] NOTICE: pid 5043, fpm_php_trace(), line 144: about to trace 36581<BR>[02-Dec-2015 09:54:39.870884] NOTICE: pid 5043, fpm_php_trace(), line 172: finished trace of 36581<BR>[02-Dec-2015 09:54:39.871187] NOTICE: pid 5043, fpm_children_bury(), line 227: child 37001 stopped for tracing<BR>[02-Dec-2015 09:54:39.871195] NOTICE: pid 5043, fpm_php_trace(), line 144: about to trace 37001<BR>[02-Dec-2015 09:54:39.871460] NOTICE: pid 5043, fpm_php_trace(), line 172: finished trace of 37001<BR>[02-Dec-2015 09:54:39.872162] DEBUG: pid 5043, fpm_got_signal(), line 76: received SIGCHLD<BR>[02-Dec-2015 09:54:39.872582] DEBUG: pid 5043, fpm_got_signal(), line 76: received SIGCHLD<BR>[02-Dec-2015 09:54:39.872950] DEBUG: pid 5043, fpm_got_signal(), line 76: received SIGCHLD<BR>[02-Dec-2015 09:54:39.873295] DEBUG: pid 5043, fpm_got_signal(), line 76: received SIGCHLD<BR>[02-Dec-2015 09:54:39.873616] DEBUG: pid 5043, fpm_event_loop(), line 424: event module triggered 1 events<BR></DIV>
<DIV><BR></DIV>
<DIV>S pozdravem<BR>Branislav Viest<BR><BR></DIV><BR>
<DIV>
<HR>
</DIV>
<DIV><B>Od: </B>"Věroslav Kaplan" <<A href="mailto:veros.kaplan@gmail.com">veros.kaplan@gmail.com</A>><BR><B>Komu: </B>"vpsFree.cz Community list" <<A href="mailto:community-list@lists.vpsfree.cz">community-list@lists.vpsfree.cz</A>><BR><B>Odoslané: </B>streda, 2. december 2015 9:48:22<BR><B>Predmet: </B>{Disarmed} [vpsFree.cz: community-list] {Disarmed} Re: php pro nginx high load<BR></DIV><BR>
<DIV>
<DIV dir=ltr>
<DIV class=gmail_default style="FONT-SIZE: small">Ahoj,</DIV>
<DIV class=gmail_default style="FONT-SIZE: small"><BR></DIV>
<DIV class=gmail_default style="FONT-SIZE: small">Tohle vypadá, že PHP-FPM z nějakého důvodu zavře to fcgi spojení. Dokážeš vytáhnout nějaké podezřelé logy z toho PHP-FPM a zkusit uhodnout, proč to dělá ?</DIV>
<DIV class=gmail_default style="FONT-SIZE: small"><BR></DIV>
<DIV class=gmail_default style="FONT-SIZE: small">--Věroš Kaplan</DIV>
<DIV class=gmail_default style="FONT-SIZE: small"><A href="http://veroskaplan.cz/">http://veroskaplan.cz/</A></DIV>
<DIV class=gmail_default style="FONT-SIZE: small">Školíme Ansible: <A href="http://ansible.cz/">http://ansible.cz/</A></DIV></DIV>
<DIV class=gmail_extra><BR>
<DIV class=gmail_quote>2015-12-02 8:25 GMT+01:00 Branislav Viest <SPAN dir=ltr><<A href="mailto:info@branoviest.com">info@branoviest.com</A>></SPAN>:<BR>
<BLOCKQUOTE class=gmail_quote style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: arial,helvetica,sans-serif">
<DIV>Ahoj,</DIV><BR>
<DIV>mám tady jeden server kde je cca 35k lidí za vteřinu. Běží tu nginx + php fpm a mysql db. V nginxu jsem rozjel microcache, což hodně ulevilo zátěži serveru. Nicméně, mám problém s PHPkem. FPM běží přes TCP (zkoušel jsem i unix socket nicméně bylo to pomalější a docházelo ke stejnému problému). Dle error logu dochází k těmto chybám:</DIV><BR>
<DIV>2015/12/01 23:26:56 [error] 27043#0: *122476 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 188.114.98.51, server: xxx request: "GET /xxx/xxxx/xxx HTTP/1.1", upstream: "fastcgi://<A href="http://127.0.0.1:9000/"><SPAN style="COLOR: red"><B>MailScanner has detected a possible fraud attempt from "127.0.0.1:9000" claiming to be</B></SPAN> <SPAN style="COLOR: red"><B>MailScanner warning: numerical links are often malicious:</B></SPAN> 127.0.0.1:9000</A>", host: "xxxx", referrer: "<A href="http://xxxxx/"><SPAN style="COLOR: red"><B>MailScanner has detected a possible fraud attempt from "xxxxx" claiming to be</B></SPAN> http://xxxxx</A>"<BR>2015/12/01 23:26:56 [error] 27032#0: *124368 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 188.114.99.50, server: xxxxxx, request: "GET /xxxxx/xxxxx HTTP/1.1", upstream: "fastcgi://<A href="http://127.0.0.1:9000/"><SPAN style="COLOR: red"><B>MailScanner has detected a possible fraud attempt from "127.0.0.1:9000" claiming to be</B></SPAN> <SPAN style="COLOR: red"><B>MailScanner warning: numerical links are often malicious:</B></SPAN> 127.0.0.1:9000</A>", host: "xxxxx", referrer: "xxxxx"</DIV><BR>
<DIV>a sype to tam docela drsně. Jak vypnu kešování v nginxu load vyskočí, třeba i na 500, těch chyb PHPka je méně ale jsou tam pořád. </DIV><BR>
<DIV>Konfigurace FPM pro pool toho webu je:</DIV><BR>
<DIV>[web1]<BR><BR>listen = <A href="http://127.0.0.1:9000/"><SPAN style="COLOR: red"><B>MailScanner has detected a possible fraud attempt from "127.0.0.1:9000" claiming to be</B></SPAN> <SPAN style="COLOR: red"><B>MailScanner warning: numerical links are often malicious:</B></SPAN> 127.0.0.1:9000</A><BR>listen.allowed_clients = 127.0.0.1<BR>listen.owner = web1<BR>listen.group = client0<BR>listen.mode = 0660<BR>listen.backlog = 65536<BR><BR>user = web1<BR>group = client0<BR><BR>request_slowlog_timeout = 5s<BR>slowlog = /var/log/php-fpm/slowlog-web1.log<BR><BR>pm = dynamic<BR>pm.max_children = 7000<BR>pm.start_servers = 4000<BR>pm.min_spare_servers = 2000<BR>pm.max_spare_servers = 4000<BR>pm.max_requests = 100<BR><BR>request_terminate_timeout = 60s<BR>rlimit_files = 500000<BR>rlimit_core = unlimited<BR>catch_workers_output = yes<BR><BR>pm.status_path = /php-status<BR><BR>chdir = /<BR></DIV><BR>
<DIV>+ openbasedir, sessions save path. atp.</DIV><BR>
<DIV>S těma hodnotama jsem se zkoušel různě hrát, nicméně to k vyřešení tohoto problému nepomohlo. Zkoušel jsem upravit i nějaké parametry jádra (local port range, tcp ack timeouty atp.) ale výsledek stejný. Dělá to i u unix socketu, což je mi právě zvláštní. Nginx konfigurace je:</DIV>
<DIV>user www-data;<BR>worker_processes 80;<BR>pid /run/nginx.pid;<BR># set open fd limit to 50000<BR>worker_rlimit_nofile 100000;<BR><BR>events {<BR>worker_connections 1024;<BR>multi_accept on;<BR>use epoll;<BR>}<BR></DIV><BR>
<DIV>ve vhostu nic zvlastniho, jenom pro php:</DIV><BR>
<DIV>fastcgi_split_path_info ^(.+\.php)(/.+)$;<BR>fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;<BR>include /etc/nginx/fastcgi_params;<BR>fastcgi_pass <A href="http://127.0.0.1:9000/"><SPAN style="COLOR: red"><B>MailScanner has detected a possible fraud attempt from "127.0.0.1:9000" claiming to be</B></SPAN> <SPAN style="COLOR: red"><B>MailScanner warning: numerical links are often malicious:</B></SPAN> 127.0.0.1:9000</A>;<BR>fastcgi_index index.php;<BR>fastcgi_intercept_errors on;<BR>fastcgi_read_timeout 4m;<BR></DIV><BR>
<DIV>Dnes bude k tomuto server nový, pro rozdělení zátěže, ale jako fakt nevím jestli to má smysl ve stejné konfiguraci, kvůli těm php chybám. Nenapadá Vás kluci, co s tím? Nebo případně návrh řešení, kterým bych nahradil tohle, při dvou serverech ? Přemýšlím nad MySQL M - M replikací, v DNS round robin a GlusterFS pro data. Ale to phpko mi nedá spát. A dnes večer to už další nápor asi nedá.</DIV><BR>
<DIV>Díky za rady a konzultace.</DIV><BR>
<DIV>S pozdravem<SPAN><SPAN style="COLOR: #888888"><BR>Branislav Viest<BR></SPAN></SPAN></DIV></DIV></DIV><BR>_______________________________________________<BR>Community-list mailing list<BR><A href="mailto:Community-list@lists.vpsfree.cz">Community-list@lists.vpsfree.cz</A><BR><A href="http://lists.vpsfree.cz/listinfo/community-list" rel=noreferrer>http://lists.vpsfree.cz/listinfo/community-list</A><BR><BR></BLOCKQUOTE></DIV><BR><BR clear=all><BR>-- <BR>
<DIV>
<DIV dir=ltr>--V.Kaplan</DIV></DIV></DIV><BR>_______________________________________________<BR>Community-list mailing list<BR><A href="mailto:Community-list@lists.vpsfree.cz">Community-list@lists.vpsfree.cz</A><BR><A href="http://lists.vpsfree.cz/listinfo/community-list">http://lists.vpsfree.cz/listinfo/community-list</A><BR></DIV></DIV></DIV>_______________________________________________<BR>Community-list mailing list<BR><A href="mailto:Community-list@lists.vpsfree.cz">Community-list@lists.vpsfree.cz</A><BR><A href="http://lists.vpsfree.cz/listinfo/community-list">http://lists.vpsfree.cz/listinfo/community-list</A><BR></DIV></BLOCKQUOTE></DIV><BR></DIV></DIV><BR></DIV>_______________________________________________<BR>Community-list mailing list<BR><A href="mailto:Community-list@lists.vpsfree.cz">Community-list@lists.vpsfree.cz</A><BR><A href="http://lists.vpsfree.cz/listinfo/community-list">http://lists.vpsfree.cz/listinfo/community-list</A><BR></DIV></BLOCKQUOTE></DIV><BR></DIV><BR></DIV>_______________________________________________<BR>Community-list mailing list<BR><A href="mailto:Community-list@lists.vpsfree.cz">Community-list@lists.vpsfree.cz</A><BR><A href="http://lists.vpsfree.cz/listinfo/community-list" rel=noreferrer>http://lists.vpsfree.cz/listinfo/community-list</A><BR></BLOCKQUOTE></DIV></DIV></BLOCKQUOTE></DIV></BODY></HTML>