29 December, 2007

performance and Qt 4.4

And again I blog about performance in KDE 4. In my last blog about that, I did express the hope the Trolls would be able to fix the issue. So, today, I tried to test the cool alien widgets which are supposed to fix flicker-when-resizing! To cut right to the chase, no, it didn't work. Blegh.

I know I'm a sucker for speed. I want my computer to wait for me, not the other way around. I mean, a machine which can easily do millions of computations a second should be fast enough to keep up with my freakingly slow brain, right? It can take up to 1/10th of a second for a signal to travel from one end of the human brain to the other - that's not even the same league as my PC is playing in... Not to mention it has two cores and 3 GB of ram to go with that insane clockspeed ;-)

So I have played with kernel patches (Con Kolivas' work is and has always been amazing in that regard). I've tried Gentoo (aw, yes, I did...). With newer GCC, X.org and linux IO and process scheduler versions, my KDE got much more smooth. Not perfect, no - but my biggest gripe, application startup speed, well - that's covered now. KDE 3.5.x didn't do badly, but if you see the difference with KDE 4 - you're gonna be surprised. Really.

Unfortunately, as I found out some time ago, drawing performance has gone through the drain in KDE 4. Now that issue is less important. Having to wait for an application to start up is annoying - you actually have to wait. If an application redraws slow, that means resizing a window won't look good. That's about it. Not a huge deal, if it wasn't for all the cool eyecandy in KDE 4 (oxygen, animations). Really, it doesn't fit very well if your apps look bloody cool but resizing causes epileptic seizures... Besides, it's not only the resizing, but also the panning of an image in Gwenview and the autoscroll feature in Konqi. The latter manages to use up to 40% of one CPU core, and the former also redraws visibly slow.

As a non-hacker, I'm not sure all issues I find are even related - but hey, one can try to figure out the issue anyway. So, here are suggested reasons for bad performance:
- Oxygen style is slow. Yep, it's a bit slower than for example QtCurve, but not much. Not THE problem.
- Qt doublebuffering is slowing everything down. Nope, disabling makes the application flicker horribly, but it won't be any faster.
- 3D stuff is slow. Sure is, but I ran the apps in KDE 3, no compositing. Not the issue.

So, now I wonder if the slowness could be fixed by the Alien Widgets, which have arrived in Qt 4.4 snapshots.

I downloaded a Qt 4.4 snapshot (today's snapshot, from 29-12-2007), compiled and installed it, and recompiled KDE with it. As I already said - no improvement, at least not as far as I could tell. Sure, with QtCurve the issue did get better, but still - if you have the KDE 3 dolphin resize almost entirely smooth, while the KDE 4 one paints visibly (I could count the number of repaints when resizing if I wanted to) - not good.

The panning in Gwenview isn't faster either, but I guess that doesn't have much to do with the alien widgets thing. I couldn't test konqi scrolling, it doesn't load webpages with Qt 4.4 ;-)

So, it's not the Alien widgets which will come to rescue KDE 4 from bad drawing performance... I see currently one possible reason for the bad speed. An anonymous reader pointed out in my last blog how Qscrollbar isn't that fast:
It seems related to QScrollBar.
Profiling konqueror in auto-scroll mode, I obtain 50% (!!) of the time spent just repainting the scrollbar.

I found no bugreports for this in TT's bugtracker (if the anonymous reader is reading this, maybe he/she should file a bug?). I'm afraid KDE 4.0 and 4.1 will have this issue - IF this gets fixed, it most likely will take until KDE 4.2 is out for this to get into the users hands. Unless the Trolls decide it's important enough for a bugfix release, of course... Personally, I hope so, this really looks bad.


Further info:
I use the following additional cmake-options in my .kdesvn-buildrc:
-DKDE4_BUILD_TESTS:BOOL=OFF -DCMAKE_BUILD_TYPE=release
-> that should lead to no debug symbols, if I understood some recent threads properly.
I've tried several styles, from the with Qt included clearlooks, QtCurve and default Oxygen. Yes, Oxygen is a bit slower with its cool background gradient, but it's not like the other ones are smooth anyway.

One last thing, of course all this might be my fault. I think I do know a little about compiling and stuff, but I'm not a code writer, and profiling is scary for me. So maybe I've made a horrible mistake, and the KDE 4.0 packages my distribution will provide soon after the release (love the rolling release schedule in Arch) will be as fast as a fox. And I don't mean FireFox, as that must be the most horribly slow painting application we have on linux (and windows and Mac OS X).

Let's hope so.

39 comments:

  1. Its rather disappointing that redraw is still slow. One of the things that makes Windows and OSX feel so much cleaner is the fast redraws. KDE3 (and GTK) redraws are pretty terrible, its sad to see that KDE4 will probably be even slower yet.

    ReplyDelete
  2. Personally, I'm fairly disgusted at Trolltech for hyping up Qt4 as "faster and lighter", when practically everyone I've spoken too has told me that e.g. resizing etc is much, *much* slower in Qt4 (there's a pair of videos around somewhere showing the same app using Qt3 and then ported to Qt4, being resized - the slowdown/ extra jerkiness is striking), and where all the micro-optimisations in memory usage are completely undone by the megabytes of extra pixmap usage caused by the (apparently necessary) double-buffering.

    It's a really bad state of affairs, and is going to damage both KDE's and Trolltech's reputations.

    Sorry to rant, but I feel really let done by both the significantly worsened performance and the fact that Trolltech won't admit there's a problem - I expect this from a closed-source company, but not from a Free Software one :(

    ReplyDelete
  3. I think that "Alien Widgets"-thing isn't in the Qt snapshot, so it may be the solution.

    bnilsen says: "I could probably tell you a lot more, but I’ll keep this blog short. Currently this work is going on in a separate research branch so it won’t show up in our snapshots, but within few weeks you should be able to try it out. I’ll blog with more details as soon as we have it integrated.

    ReplyDelete
  4. anonymous #2 again:

    "here's a pair of videos around somewhere showing the same app using Qt3 and then ported to Qt4, being resized - the slowdown/ extra jerkiness is striking"

    Here are the links:

    http://bw.uwcs.co.uk/kde3_resize.ogg
    http://bw.uwcs.co.uk/kde4_resize.ogg

    Seriously, check it out - the difference is pretty horrific :/

    NOTE: Might not play in mplayer - VLC handles it fine here, though.

    ReplyDelete
  5. @anonymous #2:
    Well, the applications start a lot faster. Very noticeably so. Thus I wouldn't say they overhyped Qt4. Also, I'm pretty sure stuff like Plasma wouldn't have been very performant (if possible at all) with Qt3. Yet the video's you post are pretty awful, and yes, they clearly show what I've been observing and blogging about lately.

    Memory usage optimizations might have been undone by the double-buffering, but I guess TT will be able to solve that on composited desktops in the future.

    cylmor:
    The current snapshots DO have the alien widgets, it has been merged with mainline a while ago.

    ReplyDelete
  6. At least it's good to know that it's not only Oxygens fault. But I hope it'll be fixed soon.

    ReplyDelete
  7. I tried pretty much everything you mentioned.

    The biggest performance improvement for me was switching from oxygen to QtCurve, but as you say it's still not exactly speedy.

    Option "UseEvents" "on" seemed to help a little too (I think this is an nvidia specific option)

    Nothing I've tried has made kde4 as fast as kde3 in terms of rendering speed. Qt4.4 appeared to make no difference at all.

    I should note that it does render much faster on my laptop (3 year old integrated intel) than on my desktop (8800GTS) so much of it is probably nvidia's driver sucking at xrender.

    ReplyDelete
  8. Thanks for being the whistleblower! I am sick of all the "QT4 rox" comments that fanboys constantly scream. Qt is a great toolkit, but the jerky performance of qt apps is not acceptable.

    ReplyDelete
  9. perhaps this will help (at least for the scollbars-repaint issue in konqueror)
    http://aseigo.blogspot.com/2006/03/better-than-watching-paint-dry.html

    ReplyDelete
  10. Is it possible that the X-server also affects the graphics performance in a negative manner?

    ReplyDelete
  11. That QT4 vs. QT3 resize thing is much too different to be an apples-to-apples comparison. It's most likely that the unfinished QT4 app is dumping lots of debugging info to stderr while redrawing. Not that this isn't flicker though; it's slow refresh speed.

    On KDE being more flickery than Win or OS X, I hear that's because Qt currently uses native X windows for *each* widget. Most toolkits do this for some reason -- maybe to catch events or to simplify canvas issues. But yeah, as I understand it, X has to send events (which have enough overhead to be capable of being on a remote machine) back and forth to each widget.

    ReplyDelete
  12. @anonymous: I think wistleblower is a bit too much credit. Sure I am concerned about this, but seeing how application startup has been improved, I'd still say Qt 4 is superior to Qt3. As I mentioned, startup time is much more important and besides, I'm sure this performance regression can and most likely will be fixed.

    As mentioned by Benji, this probably is more an issue with drivers and (lack of) acceleration than with intrinsic performance issues with Qt...

    Maybe the TT developers all have hardware with proper, open drivers ;-)

    ReplyDelete
  13. @anonymous just above my previous comment:
    The Qt4 app shouldn't be dumping debug info, it's compiled without. And as I always start KDE 4 apps from the commandline, and I don't see any output (well, a little, but not when resizing) I don't think that's the issue.

    And yes, Qt uses native X windows for each widget - but that's why I tried Qt 4.4. Qt 4.4 will introduce 'alien widgets' (read the blogs I pointed to) and it doesn't incur this X overhead anymore.

    ReplyDelete
  14. I also tried a few kde4-apps. It should be possible today, not always having the newest hardware: for windows and kde-3.X my hardware (athlon 1600, old ati radeon) works very good. But the slow scrolling of the file-listing in the latest dolphin-kde4, the scrolling and painting of thumbnails in gwenview-kde4 will keep me from upgrading to kde4-apps, until these speed-problems are fixed. I hope the reasons are found soon.

    ReplyDelete
  15. The common denominator seems to be NVidia, not Qt4, or KDE 4.
    Everyone I've seen whining about the drawing performance in kde4/qt4, are using the nvidia drivers.
    I haven't seen any slowdowns, and drawing/redrawing is actually very snappy in KDE 4, and even though I haven't done any serious testing, I would say it's even better than in KDE 3. I use the intel drivers, with an integrated intel chip.

    ReplyDelete
  16. @martin: indeed, it would seem that qt4 is just less forgiving for crappy drivers, compared to qt3. The nvidia binary drivers have been flaky forever, though the most recent releases work pretty well, unless you try to do something less common.

    I hope we either see some drastic improvements from nvidia's driver coders, or a complete working open source driver.

    ReplyDelete
  17. some comments
    * you talk about qt44 like its a release version; I suggest to put alpha or so after it to make clear its nowhere near that state.

    * I hope you wrote a mail to qt-bugs with your profiling application and results so this is on the radar. Whining on a blog doesn't really help anyone.

    * Maybe you want to compare numbers with an app like KWord that doesn't use the scrollarea directly to see if the same problem appears there.

    * If this bug is filed to be fixed for 4.4 the KDE4.0 already will benefit. Not sure why you aim for 4.2... I guess its better to be optimistic, don't you?

    ReplyDelete
  18. Re NVIDIA drivers:

    There is a way to test whether the problem lies in the crappy XRENDER performance of the NVIDIA binary drivers.

    The November beta driver release improves performance by various orders of magnitude (!). The Phoronix article

    The final release in December includes this tremendous improvements. So pretty please jos compile the new NVIDIA 169.07 Linux Display Driver into your kernel and tell us what you get!.

    If there is barely an improvement just post another comment, but if that was THE main issue I think that warrants a new blog post.

    ReplyDelete
  19. Ah! btw. don't listen to the trolling "whistleblower", qt4 crap comments and don't hesitate to continue posting your performance concerns/findings/whatever. We need people worrying about that, even if they are not developers. This here is precisely what I would call constructive criticism. You actually make the effort to measure the problem, and with some luck (NVIDIA's XRENDER?:-) we could even find what the real problem AND solution is. Now wouldn't that be awesome. People would know how to solve the problem, and you (or we) would have freed busy developers from wasting time in this crucial final sprint trying to solve it. Now that is even better than constructive criticism. That is contributing.

    ReplyDelete
  20. Whiners take note: constructive criticism does NOT mean inserting Qt/KDE/DeveloperX rox every other sentence. It means participating in solving the problem.

    ReplyDelete
  21. Opera browser is known to use Qt. I don't know why, but it "feels" very fast on Linux, compare to Firefox (which uses GTK I believe) or Konqueror for example. I am not talking about rendering performance, though that matters as well, but usages like tab switching, preferences dialog, etc. May be it is just the feeling, but I think that is what matters for average user.

    ReplyDelete
  22. @thomas:
    I will blog about the other mentioned issues, but I'll reply to you first - what else could I do, having a hamster with the same name as you have staring at me right now ;-)

    So, firstly, I was talking about an unreleased version of Qt 4.4, of course. That's why I tried a snapshot. I should've mentioned that more clearly, I guess. sorry.

    And secondly, indeed, I didn't file a bug nor profile anything - I wrote that, I wouldn't know how. That doesn't mean its all bullshit, the issues are pretty obvious (see the videos someone posted).

    Koffice currently doesn't compile, so I haven't tried it.

    And I have no idea if the issue will be fixed soon - if its big, it might be that we have to wait for the next Qt version, not a bugfix release. The next Qt version would be 4.5, I guess, and that won't be used widely in KDE before KDE 4.2 - assuming Qt also has a 6 month release schedule (and KDE sticks to it).

    If this is a qscrollthing bug, I didn't find it in the public bugtracker, so I suppose it won't be fixed soon, either.

    Of course, I will blog about the possible NVidia driver issue - as it sounds likely that's the problem.

    ReplyDelete
  23. I don't think it has to do with nvidia-drivers

    a) it also happens with ati-hardware

    b) kde3-apps work fine with the same hardware

    I think kde4 should run good/usable on a 1000 mhz pc with old 32mb graphics card like kde3 does.

    ReplyDelete
  24. Hi, FWIW, on Mac OS X, the last.fm client, which bundles some version of Qt4, is also a lot slower redrawing itslef when resizing than, say, a Finder window, even if said Finder window is displaying the new coverflow view of files.

    I'll try getting some numbers later. Maybe checking Qt4's performance on windows would be constructive too. By checking on other systems we are eliminating a whole bunch of probable places where the slowdown could be occurring. This kind of performance problem is quite difficult to measure especially because there are so many variables going under (system frameworks) and above (the app itself) Qt.

    ReplyDelete
  25. About ATI hardware having the same problem, check this. It shows that without an undocumented hidden feature introduced in a relatively recent release ATI driver's XRender performance sucks too.

    ReplyDelete
  26. I have the newest nvidia drivers and still redrawing is slower with QT4.3 then with QT3.3.8

    ReplyDelete
  27. Galactic, I think you meant this article: Hidden ATI Feature For Textured XRendering

    It says that ATI's performance still sucked in most of the XRender tests even with the new option, there was only one where it made a difference.

    It sounds like several people with Intel have said this isn't a problem for them - I'm guessing Jos doesn't have the hardware to test this out himself.

    ReplyDelete
  28. I'm having the same redraw slowness on a intel gma965 using the newish 'intel' driver and composited kwin4 with sync to vblank enabled. I'm building KDE4 from svn with kdesvn-build and I have debugging disabled (Although for certain apps I still see a lot of debug output). It is just as bad as my 7600GT (SLI) desktop system.

    Both are running a full kde4 session and kde3 apps in the session perform normally.

    ReplyDelete
  29. Andrew Stromme: how is it with compositing disabled?

    Even then, if KDE 4 apps are significantly slower than KDE 3 apps on intel hardware & open, up-to-date drivers, there goes the driver argument...

    ReplyDelete
  30. Sorry for the terribly late response. I had multiple applications due on the 1st and the 2nd, so now is really my first chance to get back to this.

    And my results are weird... very weird. No matter if compositing, or sync to vblank is enabled or diabled I still get terrible results with konqueror. This is on the kde4 and the kde3 version. It is worse than the video shown in the original blog entry here. This also happens on my desktop system, as mentioned before.

    HOWEVER, some other apps, notably konsole, are quite smooth with kde4 and kde3. There is _some_ noticeable redraw issues, but the screen is redrawing less than 15pixels later than it should be, which is much better than the 100-400 that I was experiencing with konqueror.

    Also... a related question about QImage and a drawing widget. I am playing around with a simple drawing application for kde4 where I can use the mouse to draw on a canvas (larger than the window size), and I can scroll around the canvas to see the full drawing. I'm getting major redraw issues much like the ones I'm describing here. However, applications such as KolourPaint and Krita (both in kde4 versions) do NOT have the redraw issues when scrolling the widget. I tried to look at how they were coded to see if I could copy their method of drawing and refreshing the widget, but I was having a hard time understanding... and it doesn't help that they are both huge and complex programs.

    At the moment I have implemented a custom QWidget called a canvas which draws a QImage using a QPainter to the screen when a redraw event is called for the canvas. I modeled my approach after the qt4 example of a scribblearea. However, this seems to produce a terrible response when scrolling. Any ideas? Thanks!

    ReplyDelete
  31. It looks like KDE suffers from the top-level window flicker which has now been fixed in Qt 4.4.

    It would be worth a try testing the new Qt - or first to ask at Trolltech if their fix even addresses the problem.

    ReplyDelete
  32. The flicker was fixed already, I tried an earlier snapshot. I could try the new one, see if resizing performance got better. I'm not sure, from the blog, if it's up to KDE 3.5.x speed yet, but at least it resizes without flashing :D

    ReplyDelete
  33. I'm trying out KDE 4.1 beta 1 / 2 (somewhere in between I guess, svn checkout from yesterday), and I get really bad performance from all KDE 4 programs. Konsole (and Yakuake), which both were perfectly smooth in KDE 3 are now sluggish as hell. This applies to all combinations of composite switched on and off, nvidia or nv driver, KDE's desktop effects switched on or off.

    When composite is switched on (while using the nvidia driver), one more bad thing comes up: resizing is virtually impossible. When grabbing the edge of a window and pulling it somewhere, the CPU goes to 100% load, but on the screen nothing changes. After keeping the mouse pressed for multiple seconds, maybe the window suddenly snaps to the intended new size, but sometimes the action is just being ignored.

    KDE 4 is unusable like that.

    Why is nobody commenting here anymore? What's going on?

    ReplyDelete
  34. Hi Patrick,

    I think it was concluded most performance issues were due to the bad state of proprietary video drivers like NVidia and ATI. Using good FOSS drivers, drawing performance is already much better. Still a bit worse than KDE 3.5, mostly because Qt 4.x requires much more hardware acceleration. But with the speed improvements in other areas (mostly startup of apps), I guess most consider it a good deal still...

    ReplyDelete
  35. Come on guys kde 4.1 is way too slow some things take many times more cpu then in kde3.5.x just try scroll a directory or something. is this progress? I am so so worried about the future of kde ,now it has become even slower than gtk2 aps. it is like a nightmare. try scrolling a directory in 3.5x and in 4.1 and get the shock of a lifetime in cpu usage difference.

    those that say it is ok must have some super computer or something and are too lazy to check cpu usage and compare with kde3.5, problem stuff may not be noticable on a very fast pc ,that does not make it right!
    :(

    greetings.

    ReplyDelete
  36. Like also mentioned in the http://www.kde.org/announcements/announce-4.1-rc1.php
    kde 4.1 release announcement, the slowness is an issue with the NVidia drivers. This is exactly why we released KDE 4.0 in Januari - we need third parties like NVidia to fix their stuff to work properly. KDE 4 pushes a lot of boundaries, technically speaking, and the whole stack needs to be ready for it. The issues were known for months prior to the KDE 4.0 release, but nothing was done by NV to fix them. We tried to put a bit more pressure on them by doing a release, yet still it is not fixed.

    If you want to help, complain to NVidia about the horrible performance of their drivers. We cannot do anything about it.

    ReplyDelete
  37. I have high cpu usage on "ati" open source drivers and proprietary drivers as well.
    4.1 compared to 3.5 .although usable on fast enough pc's it does not feel ok knowing this fact.


    Also the way the menu delay functions may give people an extra unnecessary feeling of slowness. Sub menus open only when you stop moving the pointer and it feels like a longer delay then on 3.5 befor it opens the sub menu. I think such settings should be made adjustable.

    ReplyDelete
  38. @last anonymous:
    The ATI drivers are better than the NVidia ones, but far from perfect. I suppose the Intel ones are the best by far, a cheap integrated Intel card peforms beautifully with KDE 4...

    About the menu delay, if there is indeed one (might be drivers, I suppose?), this sounds like something to fix. I guess you would be willing to file a bug on bugs.kde.org for kdelibs? I'll vote and comment on it, if you want.

    ReplyDelete
  39. Here is the report about the delay. http://bugs.kde.org/show_bug.cgi?id=167488


    I found some other people complaining about it here : http://dot.kde.org/1211898836/1212079110/

    greetings.

    ReplyDelete

Say something smart and be polite please!