Hongyuan Ma [Thu, 21 Jun 2018 10:14:47 +0000 (18:14 +0800)]
add results filed in TestDataSetDetailSerializer
Hongyuan Ma [Thu, 21 Jun 2018 04:54:51 +0000 (12:54 +0800)]
adapt get_dataset_info in TestRecordDetailSerializer
Hongyuan Ma [Wed, 20 Jun 2018 13:25:11 +0000 (21:25 +0800)]
add uuid field
Hongyuan Ma [Wed, 20 Jun 2018 12:32:13 +0000 (20:32 +0800)]
add hash field form TestRecord model;add TestRecordDetailViewSet;
Hongyuan Ma [Mon, 18 Jun 2018 03:39:42 +0000 (11:39 +0800)]
update basic-table
Hongyuan Ma [Mon, 18 Jun 2018 02:54:57 +0000 (10:54 +0800)]
update TestRecordListSerializer;add trend field;
Hongyuan Ma [Sun, 17 Jun 2018 11:30:54 +0000 (19:30 +0800)]
update the receiver
Hongyuan Ma [Sun, 17 Jun 2018 10:56:33 +0000 (18:56 +0800)]
add calc_status receiver
Hongyuan Ma [Sat, 16 Jun 2018 06:36:25 +0000 (14:36 +0800)]
add TestDataUploadError
Hongyuan Ma [Fri, 15 Jun 2018 15:36:30 +0000 (23:36 +0800)]
add result parse in upload
Hongyuan Ma [Fri, 15 Jun 2018 15:23:01 +0000 (23:23 +0800)]
add branch_data;add cate_sn field;
Hongyuan Ma [Fri, 15 Jun 2018 12:38:48 +0000 (20:38 +0800)]
Add temporary data import
Hongyuan Ma [Fri, 15 Jun 2018 11:56:18 +0000 (19:56 +0800)]
add alias dbtool
Hongyuan Ma [Thu, 14 Jun 2018 15:00:52 +0000 (23:00 +0800)]
add CreateTestDateSetSerializer
Hongyuan Ma [Thu, 14 Jun 2018 14:28:13 +0000 (22:28 +0800)]
add CreateTestRecordSerializer
Hongyuan Ma [Wed, 13 Jun 2018 12:18:47 +0000 (20:18 +0800)]
add TestDataSet model class
Hongyuan Ma [Sat, 9 Jun 2018 05:18:14 +0000 (13:18 +0800)]
Overwrite the table component
Hongyuan Ma [Fri, 8 Jun 2018 18:14:43 +0000 (02:14 +0800)]
add machine serialler;render test list data;
Hongyuan Ma [Thu, 7 Jun 2018 14:17:34 +0000 (22:17 +0800)]
fix CORS problem;add detailInfo function
Hongyuan Ma [Wed, 6 Jun 2018 14:28:37 +0000 (22:28 +0800)]
add data upload interface
Hongyuan Ma [Mon, 4 Jun 2018 11:45:57 +0000 (19:45 +0800)]
add testDetail viewSet
Hongyuan Ma [Sun, 3 Jun 2018 12:49:50 +0000 (20:49 +0800)]
fix url problem
MaleicAcid [Sun, 3 Jun 2018 03:57:00 +0000 (11:57 +0800)]
add token auth
MaleicAcid [Sat, 2 Jun 2018 15:42:29 +0000 (23:42 +0800)]
adjust detail page layout
MaleicAcid [Sat, 2 Jun 2018 09:44:15 +0000 (17:44 +0800)]
add related serializer
MaleicAcid [Sat, 2 Jun 2018 09:29:23 +0000 (17:29 +0800)]
add TestRecord serializer
MaleicAcid [Sat, 2 Jun 2018 08:34:59 +0000 (16:34 +0800)]
adjust model class
MaleicAcid [Fri, 1 Jun 2018 15:43:41 +0000 (23:43 +0800)]
add result card component
Hongyuan Ma [Fri, 1 Jun 2018 03:58:57 +0000 (11:58 +0800)]
add result.json sample
Hongyuan Ma [Fri, 1 Jun 2018 03:57:42 +0000 (11:57 +0800)]
add detailInfo page
Hongyuan Ma [Thu, 31 May 2018 15:46:30 +0000 (23:46 +0800)]
beautify table-list
Hongyuan Ma [Mon, 28 May 2018 14:10:08 +0000 (22:10 +0800)]
update .gitignore
Hongyuan Ma [Mon, 28 May 2018 14:08:19 +0000 (22:08 +0800)]
add case sensitive in .gitignore
Hongyuan Ma [Mon, 28 May 2018 02:14:34 +0000 (10:14 +0800)]
adjust rate-bar color
Hongyuan Ma [Mon, 28 May 2018 01:38:01 +0000 (09:38 +0800)]
add status-service
Hongyuan Ma [Mon, 28 May 2018 01:24:42 +0000 (09:24 +0800)]
add wating state for applyBtn
Hongyuan Ma [Sun, 27 May 2018 12:41:14 +0000 (20:41 +0800)]
accomplist table-list
Hongyuan Ma [Sat, 26 May 2018 04:19:57 +0000 (12:19 +0800)]
accompilsh selectItem function
Hongyuan Ma [Fri, 25 May 2018 16:26:36 +0000 (00:26 +0800)]
update rate-bar
Hongyuan Ma [Thu, 24 May 2018 15:52:52 +0000 (23:52 +0800)]
add RateBat component
Hongyuan Ma [Thu, 24 May 2018 12:33:04 +0000 (20:33 +0800)]
add pg font icon
Hongyuan Ma [Wed, 23 May 2018 14:12:46 +0000 (22:12 +0800)]
update clearBtn function
Hongyuan Ma [Wed, 23 May 2018 13:54:10 +0000 (21:54 +0800)]
add table util
Hongyuan Ma [Wed, 23 May 2018 13:40:03 +0000 (21:40 +0800)]
add ppolicy page
Hongyuan Ma [Tue, 22 May 2018 13:40:56 +0000 (21:40 +0800)]
add deleteItem func;
Hongyuan Ma [Tue, 22 May 2018 13:03:14 +0000 (21:03 +0800)]
add selected items css
Hongyuan Ma [Tue, 22 May 2018 12:46:40 +0000 (20:46 +0800)]
add selectItemClick function;
Hongyuan Ma [Mon, 21 May 2018 08:33:02 +0000 (16:33 +0800)]
add general pagination
Hongyuan Ma [Sun, 20 May 2018 14:52:09 +0000 (22:52 +0800)]
fix map issue
Hongyuan Ma [Sun, 20 May 2018 14:27:58 +0000 (22:27 +0800)]
Separate filter conditions
Hongyuan Ma [Fri, 18 May 2018 15:06:24 +0000 (23:06 +0800)]
add hover effect
Hongyuan Ma [Fri, 18 May 2018 14:47:07 +0000 (22:47 +0800)]
add loggedin dom
Hongyuan Ma [Thu, 17 May 2018 14:30:23 +0000 (22:30 +0800)]
add util.js
Hongyuan Ma [Thu, 17 May 2018 14:14:12 +0000 (22:14 +0800)]
add login page
Hongyuan Ma [Thu, 17 May 2018 13:21:28 +0000 (21:21 +0800)]
update readme.rst
Hongyuan Ma [Thu, 17 May 2018 13:15:58 +0000 (21:15 +0800)]
Fix reference issues
Hongyuan Ma [Wed, 16 May 2018 10:53:56 +0000 (18:53 +0800)]
modify filter component
Hongyuan Ma [Tue, 15 May 2018 15:13:27 +0000 (23:13 +0800)]
modify settings.py
Hongyuan Ma [Tue, 15 May 2018 14:49:46 +0000 (22:49 +0800)]
add apps,db_tools,extra_apps folders
Hongyuan Ma [Sun, 13 May 2018 14:24:04 +0000 (22:24 +0800)]
online adapter
Hongyuan Ma [Sun, 13 May 2018 14:21:19 +0000 (22:21 +0800)]
online adapter
Hongyuan Ma [Sun, 13 May 2018 14:17:14 +0000 (22:17 +0800)]
online adapter
Hongyuan Ma [Sun, 13 May 2018 12:28:27 +0000 (20:28 +0800)]
add front-end code
Mark Wong [Sat, 10 Feb 2018 01:59:44 +0000 (17:59 -0800)]
Refactor pgbench results output
The idea is that one results document represents a test of one
particular commit. Then it contains the results of all the tests run
against that commit. I think it makes more sense to restructure the
pgbench results to be like:
{
"pgbench": {
"ro": {
"results at a scale factor": {
"number of clients": {
"std": 4389.
727756305762,
"metric": 41390.590287,
"median": 43137.716637,
"results": [
{
... pgbench results of a test ...
}
...
]
},
....
},
"rw": {
... read write test results ...
},
... other tests results
}
Mark Wong [Tue, 6 Feb 2018 19:19:09 +0000 (11:19 -0800)]
Update default pgbench and postgresql settings
Make the default settings what we want to be tested. The idea is to
minimize setup on the clients.
Mark Wong [Wed, 13 Dec 2017 19:54:43 +0000 (11:54 -0800)]
Use collectd for system and postgres statistics
Mark Wong [Sat, 3 Feb 2018 22:15:19 +0000 (14:15 -0800)]
Move results directory check to start of client
Between the benchmark and collector modules, it's not clear who might
have created the output directory first. Leave a warning in the
benchmark modules for now, otherwise don't let it prevent the tests from
running.
Mark Wong [Thu, 1 Feb 2018 19:17:16 +0000 (11:17 -0800)]
Use https instead of git protocol
This is for ease of automated testing. Using git:// can require manual
confirmation about updating the ssh known hosts file.
Mark Wong [Wed, 31 Jan 2018 22:13:31 +0000 (14:13 -0800)]
Add support for FreeBSD
The memory calculations for pgbench will need to be platform specific.
Also the system data collection may have to be platform specific. Thus
far, just added conditions so that things will run on FreeBSD now.
Mark Wong [Wed, 31 Jan 2018 00:04:25 +0000 (16:04 -0800)]
Set default directories to /tmp
Set the default directories in the client settings.py file to use /tmp.
This will make it easier to automate testing by reducing the number of
settings to configure.
Mark Wong [Tue, 9 Jan 2018 17:15:08 +0000 (09:15 -0800)]
Set cwd when running pgbench
The output directory is used for the cwd when executing pgbench so that
the output files are saved in the results directory.
Mark Wong [Wed, 13 Dec 2017 19:51:43 +0000 (11:51 -0800)]
Update ignore file for ctags
Mark Wong [Wed, 13 Dec 2017 19:31:16 +0000 (11:31 -0800)]
Add sbin locations to PATH for sysctl
The location of sysctl may change depending on the distribution. Try to
add all possible paths to the environment.
Mark Wong [Wed, 27 Sep 2017 21:00:10 +0000 (14:00 -0700)]
Add requirements.txt for the client
Mark Wong [Mon, 7 Aug 2017 23:10:18 +0000 (16:10 -0700)]
Force csv.DictWriter to create unix file format
Mark Wong [Sun, 30 Jul 2017 23:19:35 +0000 (16:19 -0700)]
Always write results of a test to results.json
The idea is to keep things simple by having a directory of results per
test, and to have each directory with a consistently named results
files.
Mark Wong [Sat, 29 Jul 2017 00:51:05 +0000 (17:51 -0700)]
Capture uname information
It has some things that are not in the sysctl information. Like
hardware architecture.
Mark Wong [Fri, 28 Jul 2017 00:06:06 +0000 (17:06 -0700)]
Stop storing log files as json
Keep the json results document small by saving log information
externally. This include:
* postgres log
* pgbench output
* sar output
Mark Wong [Thu, 27 Jul 2017 00:51:59 +0000 (17:51 -0700)]
Fix use of env in call()
The env parameter to call() was clearing the PATH environment to only
have the postgres binary install path. Modify only the PATH
environment.
Mark Wong [Mon, 24 Jul 2017 04:50:14 +0000 (21:50 -0700)]
Install pgbench from contrib for older versions
Prior to 9.5, pgbench was in contrib.
Mark Wong [Mon, 24 Jul 2017 01:45:06 +0000 (18:45 -0700)]
Alter postgres options on the command line
This allows testing of older versions of postgres. Otherwise only
versions of postgres supporting ALTER SYSTEM (9.4+) will work with this
client.
Mark Wong [Fri, 28 Jul 2017 16:51:34 +0000 (09:51 -0700)]
Saved timestamp from test start
Mark Wong [Fri, 28 Jul 2017 16:50:14 +0000 (09:50 -0700)]
Use python2.7 from environment settings
Mark Wong [Tue, 18 Jul 2017 20:55:51 +0000 (13:55 -0700)]
Pythonic way to check for None
Also check to make sure files exist before trying to collect them.
Mark Wong [Tue, 18 Jul 2017 19:44:14 +0000 (12:44 -0700)]
Fix check for csv flag
Mark Wong [Tue, 18 Jul 2017 18:00:24 +0000 (11:00 -0700)]
Fix handling of csv option
Mark Wong [Thu, 20 Jul 2017 16:38:15 +0000 (09:38 -0700)]
pep8 coding style
Tomas Vondra [Thu, 13 Oct 2016 13:39:10 +0000 (15:39 +0200)]
also add CSV output to pgbench
Tomas Vondra [Thu, 13 Oct 2016 12:48:03 +0000 (14:48 +0200)]
add support for collecting results into simple CSV file
Originally the results were collected into a JSON format, which is
not very convenient when running ad-hoc benchmarks.
Tomas Vondra [Thu, 13 Oct 2016 12:27:25 +0000 (14:27 +0200)]
add shebang explicitly invoking Python 2.7
The code is not fully compatible with Python 3.x (although it should
not be a big deal to make that happen).
Tomas Vondra [Thu, 13 Oct 2016 11:47:33 +0000 (13:47 +0200)]
explicitly flush stdout in log()
Otherwise the messages may get buffered for a quite long time, which
is annoying, particularly in multi-process system.
Tomas Vondra [Thu, 13 Oct 2016 11:41:43 +0000 (13:41 +0200)]
fix lockups in PostgreSQL collector
When the workers produce results and return them through a queue, we need
to first drain the queue and then wait for the thread to join(). Otherwise
it seems to lock up - the join() call with wait as long as there are items
in the queue. This also happens because we do both things (reading from
queue and waiting for the thread) in the same thread.
Tomas Vondra [Thu, 13 Oct 2016 11:22:49 +0000 (13:22 +0200)]
open connections in a way resilient to temporary failures
In case of connection failure (e.g. a backend getting terminated by
OOM killer and taking down the whole server), the workers failed
with an exception without putting a result into the queue, leading
to hangs.
This fixes that by making the workers resilient to connection issues
by catching the exceptions and terminating nicely. Also, we retry
connection several times, to continue with benchmarking even after
OOM hits us unexpectedly, etc.
Tomas Vondra [Thu, 11 Aug 2016 00:00:32 +0000 (02:00 +0200)]
Check configuration before running any benchmarks
Before doing any heavy-lifting, check existence of directories,
binaries and some other sanity checks.
Tomas Vondra [Thu, 11 Aug 2016 00:08:06 +0000 (02:08 +0200)]
Fix CWD when running 'git pull', fix parameters.
When refreshing a reporitory clone, running 'git pull $dir' does
not do the trick - it assumes the CWD is the clone, and imports
data from repository at $dir. So we need to set CWD properly and
do just 'git pull'.
Also, prefix class members with 'self'.
Tomas Vondra [Wed, 10 Aug 2016 22:58:43 +0000 (00:58 +0200)]
Add .lock to .gitignore
Tomas Vondra [Wed, 10 Aug 2016 22:46:26 +0000 (00:46 +0200)]
Rework the perffarm client configuration.
Instead of configuration hardcoded into the perffrarm-client.py
script, move it into a separate settings.py file, and allow
override using settings_local.py (not required).
Also somewhat improve the benchmark configurations by allowing
specifying number of runs, durations etc. in the configuration
file (instead of using default parameter values).
Tomas Vondra [Wed, 10 Aug 2016 22:06:11 +0000 (00:06 +0200)]
Handle missing sar stats gracefully
Tomas Vondra [Wed, 10 Aug 2016 21:23:55 +0000 (23:23 +0200)]
Import initial version of the client
Dave Page [Tue, 15 Mar 2016 11:36:06 +0000 (11:36 +0000)]
Update requirements.txt