tag:blogger.com,1999:blog-7476449567742726187.post105543878505497488..comments2023-12-18T12:25:52.296-08:00Comments on Database Soup: We need a webapp benchmarkJosh Berkushttp://www.blogger.com/profile/09671139717468724246noreply@blogger.comBlogger23125tag:blogger.com,1999:blog-7476449567742726187.post-36587839889840070392014-12-07T14:17:35.934-08:002014-12-07T14:17:35.934-08:00Came across this, perhaps it has something useful:...Came across this, perhaps it has something useful:<br /><br />https://github.com/oltpbenchmark/oltpbenchChrishttps://www.blogger.com/profile/14478430847581027628noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-45015001510692442022014-11-20T23:23:23.744-08:002014-11-20T23:23:23.744-08:00You can use something like tcpcopy: https://github...You can use something like tcpcopy: https://github.com/session-replay-tools/tcpcopy & a real log from a Rails/Django MySQL intercept: https://github.com/session-replay-tools/mysql-replay-module<br /><br /><br />Note: I haven't used it yet, but seems promising!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-17013825122583902602014-11-14T09:54:37.462-08:002014-11-14T09:54:37.462-08:00If I end up writing my own benchmark, I'd want...If I end up writing my own benchmark, I'd want to hold of on pagination until the 2nd version. It's definitely a real problem, but it's also very hard to set up paginated data in a synthetically generated benchmark.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-35507192323510094782014-11-12T14:19:54.020-08:002014-11-12T14:19:54.020-08:00I think it's a great idea. One thing to add to...I think it's a great idea. One thing to add to the list: common pagination techniques, with abusing limit/offset being number 1. That's where most of my unexpected performance variation (and corresponding headaches) comes from.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-64317514168132602852014-11-10T09:14:08.796-08:002014-11-10T09:14:08.796-08:00Um ... yeah. You do that. Post a link when you h...Um ... yeah. You do that. Post a link when you have something.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-33376722618808518992014-11-10T08:23:09.161-08:002014-11-10T08:23:09.161-08:00Yeah. Conventional web benchmarks are really meas...Yeah. Conventional web benchmarks are really measuring the latency among javascript, the webserver, and the database server. We need not do it that way any more. At least for a significant number of application types. HTTP was always a bad paradigm for OLTP applications, and with current day infrastructure, we can do away with the disconnected-ness.<br /><br />Building a benchmark for yesterday's paradigm only serves to support yesterday's paradigm.Robert Younghttps://www.blogger.com/profile/09056808374481236610noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-7591468503953079972014-11-09T20:35:46.510-08:002014-11-09T20:35:46.510-08:00Lawrence: you seem to have an axe to grind which h...Lawrence: you seem to have an axe to grind which has little to do with my post. No thanks.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-62580814767702758862014-11-09T20:31:22.953-08:002014-11-09T20:31:22.953-08:00Thanks, Baron! However, I'm not looking to ma...Thanks, Baron! However, I'm not looking to mainly benchmark graph performance.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-27966007468988838842014-11-09T00:55:16.119-08:002014-11-09T00:55:16.119-08:00I'm not convinced your suggested set of measur...I'm not convinced your suggested set of measurements reflects typical web apps. <br />* Server-side sessions are rare in my experience, signed or encrypted cookies being the preferred alternative for almost any situation. (Sites with server-side sessions always seem frustratingly unusable with the session timing out at inopportune times, the implementation against the grain of HTTP.)<br />* "likes" should generally be implemented asynchronously, perhaps via a webserver log which is read in in small batches, so a 1 like per transaction under load would be unusual.<br />* The login action needs to reflect a usual user table rather than a database user. Very, very few webapps map app users to db users.Laurence Rowehttps://www.blogger.com/profile/02164563755538372331noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-92122617587565275322014-11-08T17:55:40.499-08:002014-11-08T17:55:40.499-08:00I think you should look into linkbench.I think you should look into linkbench.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-24934521151225393772014-11-07T15:39:31.285-08:002014-11-07T15:39:31.285-08:00The structure isn't optimal as it's design...The structure isn't optimal as it's designed to be generic both workload and database wise. The good stuff is usually at workloads/*/procedures/*.javaArthur Silvahttps://www.blogger.com/profile/16750289822317871591noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-12099939986638409542014-11-07T15:19:41.282-08:002014-11-07T15:19:41.282-08:00Yeah, I'll need to give it some testing time; ...Yeah, I'll need to give it some testing time; apparently the file structure is just very opaque. My real concern is that if I don't like some of the workloads (that is, if they're unbalanced or don't scale well), I won't be able to fix it. But test first, fix later ...Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-47219182798935579962014-11-07T15:04:35.047-08:002014-11-07T15:04:35.047-08:00What do you mean with trace file? I remember runni...What do you mean with trace file? I remember running various workloads against PG an year ago or so.Arthur Silvahttps://www.blogger.com/profile/16750289822317871591noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-72213002069782497302014-11-07T12:46:15.607-08:002014-11-07T12:46:15.607-08:00Also, sadly, the benchmarks I'm interested in ...Also, sadly, the benchmarks I'm interested in seem to be incomplete. For example, there's no Wikipedia trace file.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-29076717887436171122014-11-07T12:24:22.601-08:002014-11-07T12:24:22.601-08:00Feh. Java! Why did it have to be Java?Feh. Java! Why did it have to be Java?Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-90946997426232491422014-11-07T12:20:08.279-08:002014-11-07T12:20:08.279-08:00Um, did you have a point with this?Um, did you have a point with this?Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-37282297073586300802014-11-07T12:19:23.542-08:002014-11-07T12:19:23.542-08:00yeah, I started down the road of building a benchm...yeah, I started down the road of building a benchmark out of some common Django projects. The problem with that is unless I want to devote 5-6 webservers to running the benchmark, that becomes a benchmark *of Django* rather than of the Postges sevice.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-38060495589127420542014-11-07T12:18:16.562-08:002014-11-07T12:18:16.562-08:00Oh! Thank you! I'll dig into that, that look...Oh! Thank you! I'll dig into that, that looks like exactly what we need.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-83845891750738871792014-11-07T12:16:40.335-08:002014-11-07T12:16:40.335-08:00Yah, this is even worse than pgbench as far as bei...Yah, this is even worse than pgbench as far as being a real benchmark is concerned. No FKs, no joins. And yet uses multi-statement transactions.<br /><br />There's reasons TPC-W has been an obsolete benchmark for a decade now.Josh Berkushttps://www.blogger.com/profile/09671139717468724246noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-3558031197629890762014-11-07T10:05:30.983-08:002014-11-07T10:05:30.983-08:00In an age when we can have servers with 100s of gi...In an age when we can have servers with 100s of gigabytes of memory, terabytes of SSD, and dozens of cores for about $10K (lower end of the possibilities, yes) talking to browsers on broadband with persistent connections, perhaps we should contemplate designing apps to the VT-100/*nix database/RS-232 paradigm of times past. After all, all those so-called OO coders remain in love with dataObject/actionObject structure which is not a bit different from the COBOL/VSAM structure their granddaddies used. Some kinds of Back to The Future actually make sense; we can now do what Dr. Codd had in mind with hardware he didn't have.<br /><br />A test suite should acknowledge such a structure, rather than promoting the disconnected structure of vanilla HTTP.Robert Younghttps://www.blogger.com/profile/09056808374481236610noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-77762993770258448512014-11-07T07:29:01.459-08:002014-11-07T07:29:01.459-08:00Maybe some wiki made with Django? I'm thinking...Maybe some wiki made with Django? I'm thinking about using some existing open source project, something like https://github.com/django-wiki/django-wiki. Using Django will allow compare PostgreSql with other DBMS too.<br /><br />I think that using a framework with an ORM may be a little unfair (since it uses basic features), but will give better results, closer to more common and real workloads.Software and Motorcycleshttps://www.blogger.com/profile/05788952289316710578noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-73429950221743570812014-11-06T19:35:26.946-08:002014-11-06T19:35:26.946-08:00I think this website/project can be helpful Josh h...I think this website/project can be helpful Josh http://oltpbenchmark.com/wiki/index.php?title=Main_Page specially the Epinions and SEATS workloads.Arthur Silvahttps://www.blogger.com/profile/16750289822317871591noreply@blogger.comtag:blogger.com,1999:blog-7476449567742726187.post-9004980410425638882014-11-06T18:43:53.051-08:002014-11-06T18:43:53.051-08:00What about a spec compliant TPC-W?
Jan published ...What about a spec compliant TPC-W?<br /><br />Jan published on written in PHP several years ago it can be obtained here on pgfoundry, http://pgfoundry.org/projects/tpc-w-php/Anonymoushttps://www.blogger.com/profile/04535538708723066992noreply@blogger.com