T2 IRC Log: 2007-11-23
This is the log as captured by an IRC bot in the channel. The statements are those of the individual people and might not neccessarily reflect the policy and legal rules as set forth by the T2 SDE Project.
--- Log opened Fri Nov 23 00:00:09 2007
01:20 -!- yokoy [n=yokoy@brln-4db142a6.pool.einsundeins.de] has joined #t2
01:20 -!- yokoy_ [n=yokoy@e178095102.adsl.alicedsl.de] has quit [Read error: 104 (Connection reset by peer)]
01:22 -!- yokoy [n=yokoy@brln-4db142a6.pool.einsundeins.de] has quit [Read error: 104 (Connection reset by peer)]
01:25 -!- yokoy [n=yokoy@e178095102.adsl.alicedsl.de] has joined #t2
02:32 -!- tri [n=tri@p57ADB122.dip0.t-ipconnect.de] has quit [Read error: 110 (Connection timed out)]
04:59 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has joined #t2
05:15 -!- yokoy [n=yokoy@e178095102.adsl.alicedsl.de] has quit [Read error: 110 (Connection timed out)]
08:27 -!- pennyk_ [i=pennyk_@59.173.66.83] has joined #t2
08:33 -!- pennyk_ [i=pennyk_@59.173.66.83] has left #t2 []
08:39 -!- yokoy [n=yokoy@brln-4db80a0d.pool.einsundeins.de] has joined #t2
08:39 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has quit [Read error: 104 (Connection reset by peer)]
08:44 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has joined #t2
08:53 -!- Negher [n=negher@217.12.176.10] has joined #T2
08:54 -!- sepp [n=sepp@Z5798.z.pppool.de] has joined #t2
08:57 -!- yokoy [n=yokoy@brln-4db80a0d.pool.einsundeins.de] has quit [Read error: 110 (Connection timed out)]
09:05 -!- Negher [n=negher@217.12.176.10] has quit ["Leaving"]
09:11 -!- sepp_ [n=sepp@Z7248.z.pppool.de] has quit [Read error: 110 (Connection timed out)]
09:21 -!- Negher [n=negher@217.12.176.10] has joined #T2
09:44 -!- pennyk_ [i=pennyk_@59.173.67.142] has joined #t2
09:44 -!- sepp [n=sepp@Z5798.z.pppool.de] has quit ["Lost terminal"]
09:52 -!- CIA-12 [n=CIA@208.69.182.149] has quit [Remote closed the connection]
09:55 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
10:00 -!- CIA-12 [n=CIA@208.69.182.149] has joined #t2
10:31 < rxr> re
11:38 < CIA-12> rene * r27015 /trunk/package/develop/gwenhywfar/gwenhywfar.desc: * updated gwenhywfar (2.6.2 -> 3.0.0)
11:38 < CIA-12> rene * r27018 /trunk/package/x11/fox/fox.desc: * updated fox (1.6.30 -> 1.6.31)
11:39 < CIA-12> rene * r27016 /trunk/package/graphic/xsane/xsane.desc: * updated xsane (0.994 -> 0.995)
11:39 < CIA-12> rene * r27017 /trunk/package/base/memtester/memtester.desc: * updated memtester (4.0.7 -> 4.0.8)
11:39 < CIA-12> rene * r27019 /trunk/package/xfce4/xfce4-notes-plugin/xfce4-notes-plugin.desc: * updated xfce4-notes-plugin (1.4.1 -> 1.6.0)
11:48 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
12:30 < CIA-12> rene * r27020 /trunk/package/audio/nas/nas.desc: * updated nas (1.9 -> 1.9.1)
12:46 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit [Read error: 110 (Connection timed out)]
12:48 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
12:50 -!- Key_Gena [i=lok@gateway/tor/x-2c7a19d35dfc3359] has quit [Remote closed the connection]
13:02 -!- Key_Gena [i=lok@gateway/tor/x-74ba71f4cdf7d4bf] has joined #t2
13:16 -!- pennyk_ [i=pennyk_@59.173.67.142] has left #t2 []
13:26 < rxr> re
13:26 [Users #t2]
13:26 [@ChanServ] [ Cyda ] [ Fatal ] [ mtr ] [ TobiX ]
13:26 [ CIA-12 ] [ Dallur ] [ Key_Gena ] [ Negher ] [ valentin]
13:26 [ Codex ] [ dsoul ] [ LMJ ] [ Ragnarin] [ yokoy_ ]
13:26 [ crazy ] [ Enqlave] [ mqueiros_] [ rxr ]
13:26 -!- Irssi: #t2: Total of 19 nicks [1 ops, 0 halfops, 0 voices, 18 normal]
13:27 < mtr> hi rxr
13:38 < rxr> hi mtr
13:45 -!- mqueiros_ [n=mqueiros@c-217-70-68-246.bragatel.pt] has quit []
15:24 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
15:40 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
16:11 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has joined #t2
16:11 < tri> moin
16:14 < rxr> hi tri
16:23 < tri> hi rxr :-)
16:23 < tri> do you have same good links about linux memory managment ?
16:23 < rxr> ohem, not at ahnd
16:23 < rxr> hand
16:23 < rxr> do you mean kernel intern or user-space ?
16:24 < tri> puh - where are the differents
16:24 < tri> i know really nothing about linux memory managment
16:24 < tri> how is it done ? does it work good ? where can i see which application need with memory ?
16:24 < tri> is valgrind a good tool for this ?
16:25 < rxr> first you want to google for "linux virtual memory managment or system"
16:26 < rxr> in-kernel management is one thing, user-space in-C-library management another
16:26 < rxr> your favourite flavour of top will show the usgae
16:26 < rxr> usage
16:26 < tri> for user-space ?
16:26 < rxr> valgrind is more a tool for debugging memory access programming mistakes
16:26 < rxr> but it can also profile memory allocation / use
16:27 < rxr> a statistic of in-kernel allocated chunks:
16:27 < tri> what is this profiling option in the kernel for - also for this ?
16:27 < rxr> cat /proc/slabinfo
16:27 < rxr> there are some profiling options in the kernel - I guess you mean oprofile?
16:27 < tri> yes oprofile
16:27 < tri> hm i don't have slabinfo...
16:28 < rxr> oprofile is for something different, it's for profiling where CPU cycles are spend or other events happen, e.g. cache misses
16:28 < tri> in top i see the user-space memory managment ?
16:28 < rxr> yes, but also the size of the kernel allocated memory
16:28 < rxr> but the kernel memory not in detail
16:29 < rxr> but you really want to read some memory management introduction
16:29 < rxr> maybe at kernelnewbies or wikipedia or so
16:29 < tri> how does the in-kernel memory and the glibc memory managment compare - how does it work (please, only a very simple introduction for me :-) )
16:29 < rxr> .oO
16:30 < tri> yes, kernelnewbies is quite good - i found a lot about the kernel things there
16:30 < rxr> well - in-kernel is mostly to create a nice interfaced on top of the hardware MMU
16:30 < tri> ah ok a virtual layer over the hardware memory, right ?
16:31 < rxr> userspace C-library functions implement the programmer interface for malloc/free/realloc on top of the kernel interface
16:32 < tri> malloc etc. are c-libary functions on top of the kernel memory managment interface ?
16:34 < rxr> yes
16:34 < rxr> and the glibc, uclibc, dietlibc implementations differ
16:34 < rxr> e.g. their performance characteristic differ
16:34 < tri> ah ok - but they all use the kernel interface
16:35 < rxr> yep
16:35 < tri> there are different kernel interfaces for memory managment
16:35 < rxr> well - more or less
16:35 < tri> and is the linux kernel "good" in memory managment
16:35 < rxr> in classic Unix crap flavours user-space processes can only grow, that is allocate memory
16:36 < rxr> if they allocate 100M and then free it by the programmer the prozess will still use 100M
16:36 < rxr> the system call for getting memory is brk
16:36 < rxr> BRK(2) brk, sbrk - change data segment size
16:36 < rxr> CONFORMING TO
16:36 < rxr> 4.3BSD; SUSv1, marked LEGACY in SUSv2, removed in POSIX.1-2001.
16:37 < rxr> in this systems it's the job of the C library to keep track of free()ed areas and reuse them on later allocations by the process
16:37 < tri> ah ok
16:38 < rxr> linux allow unallocation by allocating via memory mapping areas and those mappings can be unmapped
16:38 < rxr> probably linux memory management is one of the most advanced and performing these days
16:38 < rxr> given how many people from IBM, Sgi, et al. contributed to it
16:39 < tri> is there a different between memory parts (userspace / kernelspace) and can i change what the memory is used for (how many memory a programm can allocate or how many memory is used for kernel or userspace)
16:42 < rxr> that is all of course allocated dynamically ...
16:42 < rxr> e.g. if you kernel's file-system driver needs memory it just allocated it and if it is done it frees said memory again
16:42 < rxr> same for user-space processes
16:42 < rxr> you can of course tune some caching, buffering etc. values
16:42 < rxr> but not a kernel / userspace slipt per se
16:42 < rxr> why do you ask all this stuff ?
16:44 < tri> as i told you i work in a novell netware enviroment for a few days know and they have to switch over to novell linux (that is maybe the reason why i work in this team now) - and there is a guy who really knows a lot about netware memory managment and know wants to know - who does this work in linux :-)
16:45 < CIA-12> rene * r27021 /trunk/package/office/openoffice-org/openoffice-org.conf:
16:45 < CIA-12> * further fixed openoffice-org to compile again: enable vb (visual
16:45 < CIA-12> basic) support and enable system libsndfile if present
16:45 < tri> till now i really don't ask about this because it allways works for me good
16:45 < rxr> heh :-)
16:45 < tri> but this make curious about how does it really work in linux
16:45 < rxr> actually I do nothing about network
16:46 < rxr> only that it was some half assed dos extender that annoyed the hell out of me 20y ago ...
16:46 < rxr> s/network/netware/ :_)
16:46 < rxr> 15y ago ...
16:46 < rxr> or 12?
16:46 < rxr> ok - time is not passing by as fast as I remeber .. :_)
16:46 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
16:47 < tri> so i decieded to find everything about linux kernel managment in a few weeks and then will give a short intern speech how does linux manage memory
16:47 < tri> and what are the tools to find, see or solve problems - do you know some ?
16:48 < rxr> ohm
16:48 < rxr> i mostly know debugging tools
16:48 < tri> :-)
16:48 < rxr> but they do only apply to writing bug-free programs
16:48 < rxr> not tuning whole systems
16:48 < rxr> probably (a)top is a good tool ...
16:51 < rxr> and proc output, like /proc/slabinfo that normally should be there
16:51 < rxr> and meminfo
16:51 < rxr> and maybe slubinfo in latest kernels if slub is enabled and they adapted the proc file name accordingly
16:52 < tri> ah there is also slabtop
16:52 < rxr> is that a program ?
16:52 < tri> i'm not sure it is on my box
16:52 < rxr> ah yes
16:52 < rxr> slabtop, indeed
16:53 < rxr> Active / Total Objects (% used) : 6592139 / 6789438 (97.1%)
16:53 < rxr> Active / Total Slabs (% used) : 577695 / 577716 (100.0%)
16:53 < rxr> Active / Total Caches (% used) : 109 / 185 (58.9%)
16:53 < rxr> Active / Total Size (% used) : 2097350.84K / 2138605.32K (98.1%)
16:53 < rxr> Minimum / Average / Maximum Object : 0.02K / 0.31K / 128.00K
16:53 < rxr> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
16:53 < rxr> 2893328 2878714 99% 0.23K 180833 16 723332K dentry_cache
16:53 < rxr> 1101480 1046295 94% 0.09K 27537 40 110148K buffer_head
16:53 < rxr> 647080 647080 100% 0.56K 92440 7 369760K xfs_inode
16:53 < rxr> 646970 646970 100% 0.69K 129394 5 517576K xfs_vnode
16:53 < rxr> 505640 505581 99% 0.19K 25282 20 101128K xfs_ili
16:53 < rxr> 448115 417216 93% 0.73K 89623 5 358492K reiser_inode_cache
16:53 < rxr> the numbers are probably as high as it's a 8-core, with wirtual containers and running at least 2 T2 builds, one beeing openoffice ...
16:54 < tri> ah ok you use xfs and reiserfs ?
16:54 < rxr> yes, in the past only reiserfs
16:54 < rxr> but it sucked on the sort of big RAID5 LVM volume
16:54 < rxr> so I started testing alternatives
16:54 < rxr> and so far XFS performace is quite ok
16:55 < tri> ah and there is vmtop
16:55 < rxr> so the root (/) is reiserfs and the big volume is XFS
16:55 < rxr> hm - vmtop is not on that server, nor a t2 reference build o trunk
16:55 < tri> ah ok - where does the linux memory managment have problems ?
16:55 < tri> http://www.kernel.org/pub/linux/kernel/people/mbligh/tools/vmtop
16:56 < rxr> probably only on emebedded boxes :-)
16:58 < tri> hm atop looks good
16:58 < tri> with "m" i see the memory usage of each programm
16:59 < tri> in netware there is something like a "garbage collection" - does linux have this too (i think this is in the c-lib, or ?)
17:00 < rxr> there nothing like garbage collection in the linux kernel
17:00 < rxr> also not in normal programs
17:01 < rxr> in C / C++ you need to add something garbage collection a-like manaully, like boehm-gc
17:01 < rxr> java, lua etc. come with garbage collection, though
17:01 < rxr> though linux make excessive use of unused memory for caching and such
17:02 < rxr> those mappings are re-used if the memory is required for more important tasks
17:02 < rxr> aksuch as allocations in the kernel or user processes, such as apache needing more memory etc.
17:02 < tri> ah ok so this have to be done by the programms itself - but the kernel can use this "free" memory
17:02 < tri> that sound good ?
17:03 < rxr> I think you can find something garbage collection wise in the kernel when you look in certain areas, such as network conection state which expires after some amount of time and thus freed
17:03 < rxr> but it is not like classic garbage collection of unused objects
17:03 < rxr> I haev not much negative to says bout the Linux VM
17:03 < rxr> I guess you need to surf on the SCO site to get negative comments
17:03 < rxr> probably that the Linux VM is that good because a 1:1 copy of SCO's Unix ...
17:03 < tri> wups - why ? :-)
17:04 < tri> ah ok the linux virtual memory is like scos unix memory managment
17:04 < rxr> no, I was just joking
17:04 < tri> :-)
17:04 < tri> what about memory fragmentation ?
17:05 < rxr> tri!
17:05 < tri> sorry :-)
17:05 < rxr> there are some good kernelnebie artilces, aren't there!=?!=!
17:05 < rxr> I really should do some work ...
17:06 < rxr> tri: there are different kernel allocators,
17:07 < rxr> the good old slab, the tiny for embedded use SLOB (maybe removed soon) and the new SLUB
17:07 < rxr> SLOB and SLUB have design features to work against memory fragmentation
17:07 < rxr> SLAB and SLUB, sorry
17:07 < rxr> SLOB is simply, for embedded use
17:08 < tri> ah ok this is done by the kernel itself
17:08 < rxr> well - done by the memory allocatior inside the kernel
17:09 < tri> ah ok
17:09 < rxr> though not actively, IIRC
17:09 < rxr> the allocator is just designed ot perform new allocations to prevent fragmentation
17:09 < rxr> it is not resorted at runtime IIRC
17:09 < rxr> but I think the latest kernels finally have memory pre-zeroing and background unswapping
17:10 < rxr> the former zeros unused memory when idle, so it must not be zeroed on allocation
17:10 < rxr> (thus potentially speeding up new allocations as the dirty ram is already zeroed)
17:10 < rxr> and the later loads back stuff from the SWAP when idle, historically swaped stuff would only be loaded on demand
17:11 < tri> and the used c lib (mostly glibc - or ?) is only a interface to the kernel functions - why this extra "layer" ?
17:11 < rxr> the new scheme (if it hit the mainline kernel - I did not checked recently) poteitally saves time when the formerly swapped out stuff is needed is it's already in memory again
17:11 -!- Negher [n=negher@217.12.176.10] has quit ["Leaving"]
17:11 < rxr> because the kernel level is low-level - application programmers do not want to keep track of their mappings
17:12 < rxr> they want a high level malloc and free, and also a realloc
17:12 < rxr> etc. pp.
17:12 < rxr> you really want to read some in-depth aticle with illustations and background before holding a talk :-)
17:13 < tri> hm ok i try to find some stuff online know (i now have some really good google search words - thx) and i will give you a view on my slides when i'm ready - but if my slides are shit - do you also give (paided) talks for companys ?
17:14 < tri> (i think linux memory managment is really not that easy like implement ntp, ftp or cluster serivces...)
17:18 < rxr> I cross my fingers that your slides will be good, as I barely have a free hour till 31. Dez ...
17:18 < rxr> but, yes, I'll tell you :-)
17:19 < tri> ah ok cool - and again sorry for wasting your time :-(
17:19 < rxr> btw. there was some article series with about 20 parts or so about kernel interna in the linux magazine over the past 1 1/2 year
17:19 < rxr> if you have those lingering around there also was memory management among them
17:19 < rxr> the german linux magazin that is ....
17:19 < rxr> and no I do not have all of them, I only get the magazine if they print an article written by susan or me ... :_)
17:20 < tri> yes, i haven't read this because it sounds to hm difficult... :-(
17:21 < rxr> that is there will be a "T2 als Source Distributions" Artikel in the next Linux User .-)
17:23 < tri> who cool think :-)
17:23 < tri> ah thing
17:24 < rxr> yeah - and probably more stuff comming for a special print flavour if all goes well :-)
17:25 < tri> maybe after the release there will be some more noise here :-)
17:43 < rxr> yeah - this month already will become a new top in unique visitors
17:43 < rxr> even without a 7.0-inal
17:46 < tri> ok will be back soon - cu
17:46 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has quit ["Leaving."]
18:06 < CIA-12> rene * r27022 /trunk/package/x11/rxvt-unicode/rxvt-unicode.desc: * updated rxvt-unicode (8.5 -> 8.6)
18:10 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has joined #t2
18:10 < CIA-12> rene * r27023 /trunk/package/security/libprelude/libprelude.desc: * updated libprelude (0.9.16 -> 0.9.16.1)
18:10 < tri> re
18:11 < tri> hm i have just tried to boot with my new t2 build cd - but i don't get a bash or something like this ?
18:13 < rxr> trunk, 7.0 or 6.0 ?
18:26 < tri> trunk
18:27 < rxr> hm
18:27 < rxr> no immediate idea ...
18:27 < rxr> but I plan to do way more regression testing
18:27 < tri> :-)
18:27 < rxr> preparing stuff on the 8-core in the "free minutes" for some weeks now
18:28 < rxr> next should probably be automated test image booting in qemu or so
18:28 < tri> hm do you have a idea what should i try ?
18:28 < tri> hm automated test image booting in qemu would be cool
18:29 < rxr> best post a full boot log somewhre
18:29 < rxr> but I'm out of office in some seconds
18:29 < rxr> I could take a look tomorrow morning, though
18:43 < tri> ok thx - maybe till tomorrow - cu :-)
18:58 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
20:15 -!- mqueiros_ [n=mqueiros@c-217-70-68-69.bragatel.pt] has joined #t2
21:38 -!- Key_Gena [i=lok@gateway/tor/x-74ba71f4cdf7d4bf] has quit [Read error: 104 (Connection reset by peer)]
21:44 -!- Key_Gena [i=lok@gateway/tor/x-35b65adbdb03beba] has joined #t2
22:10 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has left #t2 []
23:54 -!- mqueiros_ [n=mqueiros@c-217-70-68-69.bragatel.pt] has quit []
--- Log closed Sat Nov 24 00:00:10 2007
01:20 -!- yokoy [n=yokoy@brln-4db142a6.pool.einsundeins.de] has joined #t2
01:20 -!- yokoy_ [n=yokoy@e178095102.adsl.alicedsl.de] has quit [Read error: 104 (Connection reset by peer)]
01:22 -!- yokoy [n=yokoy@brln-4db142a6.pool.einsundeins.de] has quit [Read error: 104 (Connection reset by peer)]
01:25 -!- yokoy [n=yokoy@e178095102.adsl.alicedsl.de] has joined #t2
02:32 -!- tri [n=tri@p57ADB122.dip0.t-ipconnect.de] has quit [Read error: 110 (Connection timed out)]
04:59 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has joined #t2
05:15 -!- yokoy [n=yokoy@e178095102.adsl.alicedsl.de] has quit [Read error: 110 (Connection timed out)]
08:27 -!- pennyk_ [i=pennyk_@59.173.66.83] has joined #t2
08:33 -!- pennyk_ [i=pennyk_@59.173.66.83] has left #t2 []
08:39 -!- yokoy [n=yokoy@brln-4db80a0d.pool.einsundeins.de] has joined #t2
08:39 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has quit [Read error: 104 (Connection reset by peer)]
08:44 -!- yokoy_ [n=yokoy@e178064220.adsl.alicedsl.de] has joined #t2
08:53 -!- Negher [n=negher@217.12.176.10] has joined #T2
08:54 -!- sepp [n=sepp@Z5798.z.pppool.de] has joined #t2
08:57 -!- yokoy [n=yokoy@brln-4db80a0d.pool.einsundeins.de] has quit [Read error: 110 (Connection timed out)]
09:05 -!- Negher [n=negher@217.12.176.10] has quit ["Leaving"]
09:11 -!- sepp_ [n=sepp@Z7248.z.pppool.de] has quit [Read error: 110 (Connection timed out)]
09:21 -!- Negher [n=negher@217.12.176.10] has joined #T2
09:44 -!- pennyk_ [i=pennyk_@59.173.67.142] has joined #t2
09:44 -!- sepp [n=sepp@Z5798.z.pppool.de] has quit ["Lost terminal"]
09:52 -!- CIA-12 [n=CIA@208.69.182.149] has quit [Remote closed the connection]
09:55 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
10:00 -!- CIA-12 [n=CIA@208.69.182.149] has joined #t2
10:31 < rxr> re
11:38 < CIA-12> rene * r27015 /trunk/package/develop/gwenhywfar/gwenhywfar.desc: * updated gwenhywfar (2.6.2 -> 3.0.0)
11:38 < CIA-12> rene * r27018 /trunk/package/x11/fox/fox.desc: * updated fox (1.6.30 -> 1.6.31)
11:39 < CIA-12> rene * r27016 /trunk/package/graphic/xsane/xsane.desc: * updated xsane (0.994 -> 0.995)
11:39 < CIA-12> rene * r27017 /trunk/package/base/memtester/memtester.desc: * updated memtester (4.0.7 -> 4.0.8)
11:39 < CIA-12> rene * r27019 /trunk/package/xfce4/xfce4-notes-plugin/xfce4-notes-plugin.desc: * updated xfce4-notes-plugin (1.4.1 -> 1.6.0)
11:48 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
12:30 < CIA-12> rene * r27020 /trunk/package/audio/nas/nas.desc: * updated nas (1.9 -> 1.9.1)
12:46 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit [Read error: 110 (Connection timed out)]
12:48 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
12:50 -!- Key_Gena [i=lok@gateway/tor/x-2c7a19d35dfc3359] has quit [Remote closed the connection]
13:02 -!- Key_Gena [i=lok@gateway/tor/x-74ba71f4cdf7d4bf] has joined #t2
13:16 -!- pennyk_ [i=pennyk_@59.173.67.142] has left #t2 []
13:26 < rxr> re
13:26 [Users #t2]
13:26 [@ChanServ] [ Cyda ] [ Fatal ] [ mtr ] [ TobiX ]
13:26 [ CIA-12 ] [ Dallur ] [ Key_Gena ] [ Negher ] [ valentin]
13:26 [ Codex ] [ dsoul ] [ LMJ ] [ Ragnarin] [ yokoy_ ]
13:26 [ crazy ] [ Enqlave] [ mqueiros_] [ rxr ]
13:26 -!- Irssi: #t2: Total of 19 nicks [1 ops, 0 halfops, 0 voices, 18 normal]
13:27 < mtr> hi rxr
13:38 < rxr> hi mtr
13:45 -!- mqueiros_ [n=mqueiros@c-217-70-68-246.bragatel.pt] has quit []
15:24 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
15:40 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
16:11 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has joined #t2
16:11 < tri> moin
16:14 < rxr> hi tri
16:23 < tri> hi rxr :-)
16:23 < tri> do you have same good links about linux memory managment ?
16:23 < rxr> ohem, not at ahnd
16:23 < rxr> hand
16:23 < rxr> do you mean kernel intern or user-space ?
16:24 < tri> puh - where are the differents
16:24 < tri> i know really nothing about linux memory managment
16:24 < tri> how is it done ? does it work good ? where can i see which application need with memory ?
16:24 < tri> is valgrind a good tool for this ?
16:25 < rxr> first you want to google for "linux virtual memory managment or system"
16:26 < rxr> in-kernel management is one thing, user-space in-C-library management another
16:26 < rxr> your favourite flavour of top will show the usgae
16:26 < rxr> usage
16:26 < tri> for user-space ?
16:26 < rxr> valgrind is more a tool for debugging memory access programming mistakes
16:26 < rxr> but it can also profile memory allocation / use
16:27 < rxr> a statistic of in-kernel allocated chunks:
16:27 < tri> what is this profiling option in the kernel for - also for this ?
16:27 < rxr> cat /proc/slabinfo
16:27 < rxr> there are some profiling options in the kernel - I guess you mean oprofile?
16:27 < tri> yes oprofile
16:27 < tri> hm i don't have slabinfo...
16:28 < rxr> oprofile is for something different, it's for profiling where CPU cycles are spend or other events happen, e.g. cache misses
16:28 < tri> in top i see the user-space memory managment ?
16:28 < rxr> yes, but also the size of the kernel allocated memory
16:28 < rxr> but the kernel memory not in detail
16:29 < rxr> but you really want to read some memory management introduction
16:29 < rxr> maybe at kernelnewbies or wikipedia or so
16:29 < tri> how does the in-kernel memory and the glibc memory managment compare - how does it work (please, only a very simple introduction for me :-) )
16:29 < rxr> .oO
16:30 < tri> yes, kernelnewbies is quite good - i found a lot about the kernel things there
16:30 < rxr> well - in-kernel is mostly to create a nice interfaced on top of the hardware MMU
16:30 < tri> ah ok a virtual layer over the hardware memory, right ?
16:31 < rxr> userspace C-library functions implement the programmer interface for malloc/free/realloc on top of the kernel interface
16:32 < tri> malloc etc. are c-libary functions on top of the kernel memory managment interface ?
16:34 < rxr> yes
16:34 < rxr> and the glibc, uclibc, dietlibc implementations differ
16:34 < rxr> e.g. their performance characteristic differ
16:34 < tri> ah ok - but they all use the kernel interface
16:35 < rxr> yep
16:35 < tri> there are different kernel interfaces for memory managment
16:35 < rxr> well - more or less
16:35 < tri> and is the linux kernel "good" in memory managment
16:35 < rxr> in classic Unix crap flavours user-space processes can only grow, that is allocate memory
16:36 < rxr> if they allocate 100M and then free it by the programmer the prozess will still use 100M
16:36 < rxr> the system call for getting memory is brk
16:36 < rxr> BRK(2) brk, sbrk - change data segment size
16:36 < rxr> CONFORMING TO
16:36 < rxr> 4.3BSD; SUSv1, marked LEGACY in SUSv2, removed in POSIX.1-2001.
16:37 < rxr> in this systems it's the job of the C library to keep track of free()ed areas and reuse them on later allocations by the process
16:37 < tri> ah ok
16:38 < rxr> linux allow unallocation by allocating via memory mapping areas and those mappings can be unmapped
16:38 < rxr> probably linux memory management is one of the most advanced and performing these days
16:38 < rxr> given how many people from IBM, Sgi, et al. contributed to it
16:39 < tri> is there a different between memory parts (userspace / kernelspace) and can i change what the memory is used for (how many memory a programm can allocate or how many memory is used for kernel or userspace)
16:42 < rxr> that is all of course allocated dynamically ...
16:42 < rxr> e.g. if you kernel's file-system driver needs memory it just allocated it and if it is done it frees said memory again
16:42 < rxr> same for user-space processes
16:42 < rxr> you can of course tune some caching, buffering etc. values
16:42 < rxr> but not a kernel / userspace slipt per se
16:42 < rxr> why do you ask all this stuff ?
16:44 < tri> as i told you i work in a novell netware enviroment for a few days know and they have to switch over to novell linux (that is maybe the reason why i work in this team now) - and there is a guy who really knows a lot about netware memory managment and know wants to know - who does this work in linux :-)
16:45 < CIA-12> rene * r27021 /trunk/package/office/openoffice-org/openoffice-org.conf:
16:45 < CIA-12> * further fixed openoffice-org to compile again: enable vb (visual
16:45 < CIA-12> basic) support and enable system libsndfile if present
16:45 < tri> till now i really don't ask about this because it allways works for me good
16:45 < rxr> heh :-)
16:45 < tri> but this make curious about how does it really work in linux
16:45 < rxr> actually I do nothing about network
16:46 < rxr> only that it was some half assed dos extender that annoyed the hell out of me 20y ago ...
16:46 < rxr> s/network/netware/ :_)
16:46 < rxr> 15y ago ...
16:46 < rxr> or 12?
16:46 < rxr> ok - time is not passing by as fast as I remeber .. :_)
16:46 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has quit ["There is intelligent life out there..."]
16:47 < tri> so i decieded to find everything about linux kernel managment in a few weeks and then will give a short intern speech how does linux manage memory
16:47 < tri> and what are the tools to find, see or solve problems - do you know some ?
16:48 < rxr> ohm
16:48 < rxr> i mostly know debugging tools
16:48 < tri> :-)
16:48 < rxr> but they do only apply to writing bug-free programs
16:48 < rxr> not tuning whole systems
16:48 < rxr> probably (a)top is a good tool ...
16:51 < rxr> and proc output, like /proc/slabinfo that normally should be there
16:51 < rxr> and meminfo
16:51 < rxr> and maybe slubinfo in latest kernels if slub is enabled and they adapted the proc file name accordingly
16:52 < tri> ah there is also slabtop
16:52 < rxr> is that a program ?
16:52 < tri> i'm not sure it is on my box
16:52 < rxr> ah yes
16:52 < rxr> slabtop, indeed
16:53 < rxr> Active / Total Objects (% used) : 6592139 / 6789438 (97.1%)
16:53 < rxr> Active / Total Slabs (% used) : 577695 / 577716 (100.0%)
16:53 < rxr> Active / Total Caches (% used) : 109 / 185 (58.9%)
16:53 < rxr> Active / Total Size (% used) : 2097350.84K / 2138605.32K (98.1%)
16:53 < rxr> Minimum / Average / Maximum Object : 0.02K / 0.31K / 128.00K
16:53 < rxr> OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
16:53 < rxr> 2893328 2878714 99% 0.23K 180833 16 723332K dentry_cache
16:53 < rxr> 1101480 1046295 94% 0.09K 27537 40 110148K buffer_head
16:53 < rxr> 647080 647080 100% 0.56K 92440 7 369760K xfs_inode
16:53 < rxr> 646970 646970 100% 0.69K 129394 5 517576K xfs_vnode
16:53 < rxr> 505640 505581 99% 0.19K 25282 20 101128K xfs_ili
16:53 < rxr> 448115 417216 93% 0.73K 89623 5 358492K reiser_inode_cache
16:53 < rxr> the numbers are probably as high as it's a 8-core, with wirtual containers and running at least 2 T2 builds, one beeing openoffice ...
16:54 < tri> ah ok you use xfs and reiserfs ?
16:54 < rxr> yes, in the past only reiserfs
16:54 < rxr> but it sucked on the sort of big RAID5 LVM volume
16:54 < rxr> so I started testing alternatives
16:54 < rxr> and so far XFS performace is quite ok
16:55 < tri> ah and there is vmtop
16:55 < rxr> so the root (/) is reiserfs and the big volume is XFS
16:55 < rxr> hm - vmtop is not on that server, nor a t2 reference build o trunk
16:55 < tri> ah ok - where does the linux memory managment have problems ?
16:55 < tri> http://www.kernel.org/pub/linux/kernel/people/mbligh/tools/vmtop
16:56 < rxr> probably only on emebedded boxes :-)
16:58 < tri> hm atop looks good
16:58 < tri> with "m" i see the memory usage of each programm
16:59 < tri> in netware there is something like a "garbage collection" - does linux have this too (i think this is in the c-lib, or ?)
17:00 < rxr> there nothing like garbage collection in the linux kernel
17:00 < rxr> also not in normal programs
17:01 < rxr> in C / C++ you need to add something garbage collection a-like manaully, like boehm-gc
17:01 < rxr> java, lua etc. come with garbage collection, though
17:01 < rxr> though linux make excessive use of unused memory for caching and such
17:02 < rxr> those mappings are re-used if the memory is required for more important tasks
17:02 < rxr> aksuch as allocations in the kernel or user processes, such as apache needing more memory etc.
17:02 < tri> ah ok so this have to be done by the programms itself - but the kernel can use this "free" memory
17:02 < tri> that sound good ?
17:03 < rxr> I think you can find something garbage collection wise in the kernel when you look in certain areas, such as network conection state which expires after some amount of time and thus freed
17:03 < rxr> but it is not like classic garbage collection of unused objects
17:03 < rxr> I haev not much negative to says bout the Linux VM
17:03 < rxr> I guess you need to surf on the SCO site to get negative comments
17:03 < rxr> probably that the Linux VM is that good because a 1:1 copy of SCO's Unix ...
17:03 < tri> wups - why ? :-)
17:04 < tri> ah ok the linux virtual memory is like scos unix memory managment
17:04 < rxr> no, I was just joking
17:04 < tri> :-)
17:04 < tri> what about memory fragmentation ?
17:05 < rxr> tri!
17:05 < tri> sorry :-)
17:05 < rxr> there are some good kernelnebie artilces, aren't there!=?!=!
17:05 < rxr> I really should do some work ...
17:06 < rxr> tri: there are different kernel allocators,
17:07 < rxr> the good old slab, the tiny for embedded use SLOB (maybe removed soon) and the new SLUB
17:07 < rxr> SLOB and SLUB have design features to work against memory fragmentation
17:07 < rxr> SLAB and SLUB, sorry
17:07 < rxr> SLOB is simply, for embedded use
17:08 < tri> ah ok this is done by the kernel itself
17:08 < rxr> well - done by the memory allocatior inside the kernel
17:09 < tri> ah ok
17:09 < rxr> though not actively, IIRC
17:09 < rxr> the allocator is just designed ot perform new allocations to prevent fragmentation
17:09 < rxr> it is not resorted at runtime IIRC
17:09 < rxr> but I think the latest kernels finally have memory pre-zeroing and background unswapping
17:10 < rxr> the former zeros unused memory when idle, so it must not be zeroed on allocation
17:10 < rxr> (thus potentially speeding up new allocations as the dirty ram is already zeroed)
17:10 < rxr> and the later loads back stuff from the SWAP when idle, historically swaped stuff would only be loaded on demand
17:11 < tri> and the used c lib (mostly glibc - or ?) is only a interface to the kernel functions - why this extra "layer" ?
17:11 < rxr> the new scheme (if it hit the mainline kernel - I did not checked recently) poteitally saves time when the formerly swapped out stuff is needed is it's already in memory again
17:11 -!- Negher [n=negher@217.12.176.10] has quit ["Leaving"]
17:11 < rxr> because the kernel level is low-level - application programmers do not want to keep track of their mappings
17:12 < rxr> they want a high level malloc and free, and also a realloc
17:12 < rxr> etc. pp.
17:12 < rxr> you really want to read some in-depth aticle with illustations and background before holding a talk :-)
17:13 < tri> hm ok i try to find some stuff online know (i now have some really good google search words - thx) and i will give you a view on my slides when i'm ready - but if my slides are shit - do you also give (paided) talks for companys ?
17:14 < tri> (i think linux memory managment is really not that easy like implement ntp, ftp or cluster serivces...)
17:18 < rxr> I cross my fingers that your slides will be good, as I barely have a free hour till 31. Dez ...
17:18 < rxr> but, yes, I'll tell you :-)
17:19 < tri> ah ok cool - and again sorry for wasting your time :-(
17:19 < rxr> btw. there was some article series with about 20 parts or so about kernel interna in the linux magazine over the past 1 1/2 year
17:19 < rxr> if you have those lingering around there also was memory management among them
17:19 < rxr> the german linux magazin that is ....
17:19 < rxr> and no I do not have all of them, I only get the magazine if they print an article written by susan or me ... :_)
17:20 < tri> yes, i haven't read this because it sounds to hm difficult... :-(
17:21 < rxr> that is there will be a "T2 als Source Distributions" Artikel in the next Linux User .-)
17:23 < tri> who cool think :-)
17:23 < tri> ah thing
17:24 < rxr> yeah - and probably more stuff comming for a special print flavour if all goes well :-)
17:25 < tri> maybe after the release there will be some more noise here :-)
17:43 < rxr> yeah - this month already will become a new top in unique visitors
17:43 < rxr> even without a 7.0-inal
17:46 < tri> ok will be back soon - cu
17:46 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has quit ["Leaving."]
18:06 < CIA-12> rene * r27022 /trunk/package/x11/rxvt-unicode/rxvt-unicode.desc: * updated rxvt-unicode (8.5 -> 8.6)
18:10 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has joined #t2
18:10 < CIA-12> rene * r27023 /trunk/package/security/libprelude/libprelude.desc: * updated libprelude (0.9.16 -> 0.9.16.1)
18:10 < tri> re
18:11 < tri> hm i have just tried to boot with my new t2 build cd - but i don't get a bash or something like this ?
18:13 < rxr> trunk, 7.0 or 6.0 ?
18:26 < tri> trunk
18:27 < rxr> hm
18:27 < rxr> no immediate idea ...
18:27 < rxr> but I plan to do way more regression testing
18:27 < tri> :-)
18:27 < rxr> preparing stuff on the 8-core in the "free minutes" for some weeks now
18:28 < rxr> next should probably be automated test image booting in qemu or so
18:28 < tri> hm do you have a idea what should i try ?
18:28 < tri> hm automated test image booting in qemu would be cool
18:29 < rxr> best post a full boot log somewhre
18:29 < rxr> but I'm out of office in some seconds
18:29 < rxr> I could take a look tomorrow morning, though
18:43 < tri> ok thx - maybe till tomorrow - cu :-)
18:58 -!- Enqlave [n=stealth@sourcemage/guru/stealth] has joined #t2
20:15 -!- mqueiros_ [n=mqueiros@c-217-70-68-69.bragatel.pt] has joined #t2
21:38 -!- Key_Gena [i=lok@gateway/tor/x-74ba71f4cdf7d4bf] has quit [Read error: 104 (Connection reset by peer)]
21:44 -!- Key_Gena [i=lok@gateway/tor/x-35b65adbdb03beba] has joined #t2
22:10 -!- tri [n=tri@p57AD8752.dip0.t-ipconnect.de] has left #t2 []
23:54 -!- mqueiros_ [n=mqueiros@c-217-70-68-69.bragatel.pt] has quit []
--- Log closed Sat Nov 24 00:00:10 2007