[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: High load times with mdb

Bill MacAllister wrote:

--On Tuesday, June 25, 2013 03:10:17 PM -0700 Howard Chu <hyc@symas.com> wrote:

Probably bad default FS settings, and changed from your previous OS revision.

Also, you should watch vmstat while it runs to get a better idea of
how much time the system is spending in I/O wait.


I have just re-mkfs'ed the new, slow system to make it look like the
old, fast system.  Just to make sure nothing else changed I have
started a load on the older system.  Things look fine.


I meant mount options, mkfs should have very little impact.
ext3 journaling is pretty awful. For ext4 you probably want data=writeback mode, and you probably should compare the commit= value, which defaults to 5 seconds and also barrier; I believe the barrier default changed between kernel revisions. 


Now, comparing vmstat output, the new system is clearly badness incarnate.

Fast
====
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
  r  b swpd   free    buff  cache   si   so    bi    bo   in   cs us sy id  wa
  0  0    0 6358656 303468 9301620   0    0     0     0   45   45  0  0 100  0
  0  0    0 6358780 303468 9301620   0    0     0     0   47   41  0  0 100  0
  0  0    0 6358780 303468 9301620   0    0     0     0   47   41  0  0 100  0
  0  0    0 6358780 303468 9301620   0    0     0     0   93   43  0  1 99   0
  0  0    0 6358532 303468 9301620   0    0     0     0  141   71  0  1 99   0
  1  0    0 6358488 303468 9301620   0    0     0    14  116   48  0  1 99   0

Slow
====
procs  -----------memory----------  ---swap-- -----io---- -system-- ----cpu----
  r  b  swpd    free    buff  cache    si   so    bi    bo   in   cs us sy id wa
  1  4    0  13318088  36128 2759600    0    0     0  2134  379   83  0  0 88 12
  0  4    0  13318308  36128 2759600    0    0     0  1044  277   70  0  0 88 12
  0  4    0  13318508  36132 2759600    0    0     0   765  267   69  0  0 88 12
  0  2    0  13318240  36152 2759604    0    0     0   818  593  104  0  0 88 12
  0  2    0  13318332  36168 2759604    0    0     0  2611 1489  138  0  0 89 11

Lots of waiting, lots of blocking.  What's the deal with all that
free memory on the slow system?

I will interate on mkfs for a bit, but I thought I would send this off
incase something jumps out.

Bill




--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/