-
Notifications
You must be signed in to change notification settings - Fork 3
/
HACKING
221 lines (187 loc) · 8.06 KB
/
HACKING
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
----------------------------------------------------------------------------
Hacking Perfkit
----------------------------------------------------------------------------
Source Overview
===============
/
|
+- build/
|
| Various build system scripts and utilities required. This includes such
| things as m4 macros and autotools plugins.
|
+- perfkit-agent/
|
| The agent which runs on the target machine to manage perfkit plugins
| and child processes. The agent orchestrates *how* things happen on
| the target (as opposted to *what*). This could also be called the
| Agent.
|
+- perfkit-agent/encoders/
|
| Encoders are responsible for convering recoreded Samples and Manifests
| (which describe a series of samples) into buffers which can be decoded
| on the other end. They can provide various features such as compression
| and encryption.
|
+- perfkit-agent/listeners/
|
| Listeners provide connectivity into the agent over a form of RPC. This
| includes DBus, TCP, or another transport.
|
+- perfkit-agent/sources/
|
| Sources are the plugins into the agent which extract information from
| a single or group of processes. They, for example, could extract memory
| or network throughput and create samples to be passed on to the client.
|
+- perfkit/
|
| This directory contains the shared-library that can be used to talk to
| a perfkit-agent. It abstracts the communication protocol for the
| host process so they need not be concerned whether it is DBUS, HTTP,
| or some other protocol.
|
| The shared library provides proxy objects that represent the objects
| available on the connected agent.
|
+- tools/
|
| The tools directory contains various tools to help manage or interact
| with a target running perfkit. The perfkit-shell provides an
| interactive, readline-based, shell for controlling a agent. A
| support script is also provided that will eventually dump a significant
| amount of data on the target machine for someone to debug any problems
| that are occuring.
|
+- data/
|
| The data directory contains data files that may be installed onto the
| target host. This includes files such as DBUS introspection xml.
|
+- cut-n-paste/
|
| The cut-n-paste directory contains files that we consume in perfkit that
| can be copied verbatim from other source trees. It is desired to keep
| these files unchanged so the process of updating them is as simple as
| dropping in a new file.
|
+- tests/
|
| This directory contains automated tests that are written using the GLib
| testing framework. They are meant to be run before checking in new code
| within critical paths.
|
perfkit-agent
==============
The perfkit agent is a process which runs on a target system that manages
the profiling or systems monitoring. It can spawn processes to profile
as well as profile existing ones. It provides an RPC mechanism to control
the agent remotely over such transports as DBUS.
Data sources, which provide profiling instrumentation, are provided through
perfkit-agent plugins. Good examples of a data sources would be memory
profilers, network bandwidth monitors, or cpu recorders.
Data sources, when recording, create "Samples". Samples are nuggets of
information about the inferior process (the target). They can contain
arbitrary information and are backhauled to interested clients through
their aggregation channel.
A rough visual pictogram would look like:
{client} --- {DBUS,...} --- {Agent} --- {Channel} --- {Source,...}
In this case, the client is connecting to the perfkit-agent over DBUS.
Inside the agent exists a Channel which has various Sources attached
to it. When those data sources yield new instrumentation samples, they
are delivered to the Channel who then aggregates them. The client can
receive the channels and is free to do with them what it desires.
For example, it might be interesting to simply log the samples to a file
for later analysis. A simple file format could be constructed to record
the samples as they come off the logical "wire" and stored to disk.
libperfkit-1.0
==============
Libperfkit is a shared library that reduces the complexity in talking to
a remote perfkit-agent. It provides a transport abstraction so the
physical transport used is not the problem of the client. The standard
connection used is DBUS for local profiling. You can use the library
to create a new connection as such:
PkConnection *conn = pk_connection_new_from_uri ("dbus://");
Currently, only DBUS is supported.
You have two ways of using libperfkit. You can use the lowlevel API
which is lightweight and direct. Alternatively, you can use the object
model API which creates proxy objects for the remote agent. This might
be more friendly to GUI developers as GObject properties can be binded
to Gtk objects like GtkCellRenderers.
The basic format of the shared library follows the semantics of the
perfkit-agent.
/ PkConnection
|
| Abstracts the transport protocol used to talk to the agent.
| Currently, DBUS is supported. Provides all RPCs via the
| <perfkit/perfkit-lowlevel.h> API.
|
+- PkManager
|
| Highlevel abstraction for managing a agent on the opposite end-point
| of the connection. Gives access of locating objects and various
| accounting information.
|
+--- PkChannel
|
| Proxy for an individual channel found on the remote perfkit-agent.
| Provides access to manipulate the channel. Also can start, stop,
| pause, unpause, and receive samplse from the channel.
+--- PkSource
|
| Proxy for an individual source found on the remote perfkit-agent.
| Provides access to manipulate the source.
Profiling sample delivery is a critical path in the lifecycle of the
perfkit-agent and client. Perfkit, by default, uses the DBUS for
communicating with clients. The particular DBUS connection used is managed
by the DBUS agent and therefore has a decent amount of overhead. Therefore,
a private connection between the client and the agent is created and the
DBUS serialization protocol is used in a peer-to-peer manner.
perfkit-shell
=============
Perfkit-shell is an interactive, readline-based shell. It is pretty strait
forward and uses EggLine to provide the shell interaction. EmtFmt is used
to format console based text where appropriate so that it is easy to read.
The shell uses libperfkit to communicate to the perfkit-agent. The goal
is for the shell to be able to talk to remote servers as well as the local
agent.
perfkit-support
===============
Perfkit-support provides a support tool for administrators to dump
information about a malfunctioning perfkit-agent for post analysis.
Ideally, it will include any necessary core-dumps, information about
running processes, their children, loaded libraries, logs, and other
troubleshooting information.
----------------------------------------------------------------------------
Roadmap
----------------------------------------------------------------------------
Milestone I.
============
* Spawning a new process with arguments. [DONE]
* Ability for data sources to alter environment variables prior to process
spawning.
* Data sources for:
- System Memory [DONE]
- System CPU
- GLib Memory Allocations
- GLib signal emissions
* Ability to set sampling frequencies. This is done per source.
* Console shell to manipulate commands and monitor on the fly.
* Start on user interface using Gtk with plugins using Ethos.
* Initial libperfkit API [MOSTLY DONE].
Milestone II.
=============
* Export input/output stream for spawned processes over DBUS.
* Data sources for:
- Power consumption
- Network traffic
- Gtk/Gdk events
* API Review.
* Security Review.
* Python bindings.
* Lots more data sources.
Milestone III.
==============
* Start stablizing API.
* Start implementing security.