· 9 years ago · May 18, 2017, 05:10 PM
1ATI :: Data collection scripts
2==============================
3
4Purpose ::
5
6The Automated Test Infrastructure (ATI) is a collection of Perl
7scripts that interact with the Ixia testing software and devices to
8make testing your network faster, easier, and more accurate. These
9goals are completed by eliminating the need to tediously enter
10potentially thousands of commands by hand.
11
12A web based reporting tool compliments the data collection scripts
13making network differences clear and concise. The data collected using
14the Perl scripts is stored both in sql and cvs format. The reporting
15tool uses these dynamically created databases to generate information
16tables on the fly, as well as saving customized canned reports live
17during the tests. Command responses can also be verified by viewing
18the saved raw XML output.
19
20 Tar ball contents ::
21
22After gunzip and unpacking the ATI.tar.gz file you will find two main
23folders. The lib folder and the scripts folder. The lib folder holds
24several custom perl modules required for the perl scripts. In the
25scripts folder you'll find the actual scripts, logs, and configuration
26files.
27
28lib/TPG:
29
30 - ATI.pm
31
32 - IXIA.pm
33
34 - Test.pm
35
36 - Networkbaseline.pm
37
38
39scripts/tpg_development:
40
41 - tpg_DB_query.pl
42
43 - tpg_develBase_XX.pspec
44
45 - tpg_develBase_XX.pl
46
47 - tpg_ixiaBgpSessions_XX.csv
48
49 - tpg_ixiaIgmpGroups_XX.csv
50
51 - tpg_ixiaProtoInts_XX.csv
52
53 - tpg_ixiaStreams_XX.csv
54
55 Requirements ::
56
57MySQL server
58
59Both the web based reporting tools and the Perl collection scripts
60require MySQL server 5.1 or later to be installed on the server. All
61privileges should be granted to the database user which the Perl
62scripts are using -- the PHP reporting tools only require viewing
63privileges.
64
65It is important that the MySQL max packet size be configured to 1GB to
66prevent data loss and corruption. You can do this by editing the
67my.cnf so that max_allowed_packet=1000M or you can run the following
68commands from the MySQL shell to set and verify the max allowed packet
69size:
70
71 *mysql> mysql --max_allowed_packet=1000M;
72
73 *mysql> show variables like `max_allowed_packet';
74
75
76HTTP server with PHP support
77
78The WebUI reporting tools also require a web sever with PHP. It is
79recommended that you compile PHP 6 from source as it includes a built
80in mysql client -- this will help avoid any potential library
81conflicts. You can do this by using the --with-mysql=mysqlnd option
82when compiling PHP 6:
83
84 ./configure —with-mysql=mysqlnd
85
86 make
87
88 make install
89
90Perl Modules
91
92In order for the Perl scripts to run properly, you will need to ensure
93that the following modules and their dependencies are installed on the
94server running the scripts:
95
96 - Tcl
97
98 - Symbol
99
100 - POSIX
101
102 - Net::SSH2
103
104 - Data::Dumper
105
106 - Term::ReadKey
107
108 - Net::Netconf::Manager (Supplied by Juniper and requests Perl 5.6.1 or later)
109
110 - IPC::Open2
111
112 - Carp
113
114 - Cwd
115
116 - XML::Simple
117
118 - XML::RegExp
119
120 - IO::Pty
121
122 - DBI
123
124 - DBD::mysql
125
126 - Statistics::Descriptive
127
128 Configuring scripts ::
129
130Pointing scripts to TPG perl modules
131
132There are also several TPG modules which come packaged with the script
133files. These are found in the "lib" folder when you unpack the scripts
134(please see above section labeled `Tar ball contents'). The main perl
135script needs to be edited to point to the path of these modules.
136
137The script that needs to be edited is called `tpg_develBase_XX.pl'
138where XX is the version number of the script. At the very top of this
139script you will find a line that looks like the following:
140
141 use lib qw(/home/elombera/devel/lib);
142
143This line tells the script where additional libraries can be found. In
144order for the script to load these modules you simply change the path
145to point to where your TPG lib folder can be found.
146
147Test parameters
148
149The test parameters can be found and set in the
150`tpg_develBase_XX.pspec' file. These test parameters contain
151information the scripts need to connect to the ixia server and the
152ixia chassis. They also set which csv files to load from the scripts
153folder and which database to enter test results into.
154
155Parameters example :
156
157 $test_parameters = {
158
159 Username => "install",
160
161 Password => "install",
162
163 Target_database_Name => `tpg_stfi_validation_003',
164
165 TC_NGDC_FR_01_timer_a => `30',
166
167 Config_ixia_tcl_server_IP => `172.16.1.11',
168
169 Config_ixia_chassis_IP => `172.16.1.21',
170
171 Config_ixia_version => `4.10.250.23',
172
173 Config_ixia_LibPath => `/usr/local/lib/ixia/lib',
174
175 Generate_tcl_outputfile => `1',
176
177 FileName_IxiaProtoInts => `tpg_X_ixiaProtoInts_002.csv',
178
179 FileName_IxiaStreams => `tpg_X_ixiaStreams_002.csv',
180
181 FileName_IxiaBgpSessions => `tpg_X_ixiaBgpSessions_003.csv',
182
183 FileName_IxiaPimSessions => `tpg_X_ixiaPimSessions_002.csv',
184
185 FileName_IxiaIgmpGroups => `tpg_X_ixiaIgmpGroups_002.csv',
186
187 };
188
189
190Adding Devices
191
192After the test parameters there is a section in the
193`tpg_develBase_XX.pspec' file for devices. Here you can define a long
194list of devices for the scripts to collect data from.
195
196To add a new device -- define the host name of the device (eg.
197MAH-M1-CPR-01), the type the device falls under (please see defining
198device types and classes after this section), the Original equipment
199manufacturer, the Management IP, the class, and finally the connection
200type for the router. Below you can see examples of two devices added
201to the device section.
202
203Device example:
204
205 $devices = {
206
207 "MAH-M1-CPR-01" => {
208
209 type => `router',
210
211 oem => `juniper',
212
213 MgmtIP => `172.16.1.100',
214
215 class => `PeeringRouter',
216
217 connectionType => `netconf',
218
219 },
220
221 "MAH-M2-CPR-01" => {
222
223 type => `router',
224
225 oem => `juniper',
226
227 MgmtIP => `172.16.1.102',
228
229 class => `PeeringRouter',
230
231 connectionType => `netconf',
232
233 },
234
235 };
236
237
238- Custom device types and classes
239- How to configure custom tests
240- Ixia cvs file usage
241
242 Using data collection scripts ::
243
244- Establish a baseline
245- Running the main .pl script
246
247 Using reporting tools ::
248
249The reporting tools have two main modes of use. First, they can be
250generated live during the testing procedures -- as data is collected
251and then parsed, it can also be fed to a reporting script to generate
252canned reports to be viewed later. Second, you can use them to look at
253specific pieces of data through the easy to use WebUI.
254
255Viewing detailed reports
256
257These are the reports that allow you to get the most specific
258information from a device. In order to view a report at this level you
259must specify the Database, Test run, Device, Command, first, and
260second designators from that test run.
261
262Once these selections are made from drop down boxes, you will be
263presented with a series of tables with information about the command
264you specified. The information from one designator will be displayed
265next to the information from the second designator. If the table
266element is light blue, there are no changes from designator one to
267two. However, if the table element shows up as red, this means the
268information has changed.
269
270Sometimes data may not appear the way you expect or you will need more
271information from the device you are analyzing. In order to get
272detailed information from the device you are analyzing you can click
273the "View Raw Data" button. This will generate a large table with the
274raw XML output from the router and the two designator points.
275
276Example of selections:
277
278 Database: sfti_validation_010
279
280 Test run: Mon Oct 12 07:55:06 2009
281
282 Device: MAH-M1-CPR-01
283
284 Command: get_interface_information
285
286 First Designator: Baseline Collection
287
288 Second Designator: After link failure
289
290Static device output reports
291
292One of the static reports that can be generated creates a report for
293the baseline of every command ran on every device. These are also
294saved into an individual file. Although these reports do not show data
295differences from one test designator to another, they are useful to
296verify that data is being entered the way you expect into the database
297and reporting tools. You can quickly and easily browse these reports
298to ensure that data for commands and devices are accurate.
299
300In order to generate these reports you must first run a data
301collection cycle that includes a baseline designator. Then you will
302have to change the `config_device.php' file to point to the database
303for the test run you wish to verify. You can do this by changing the
304following line, located at the top of the file, to point to the
305correct database name:
306
307 $dbname = "sfti_validation_011";
308
309Once you have done this, you will need to open the `report_device.php'
310file in your web browser. This will output the names of the reports as
311they are created. You can then view these files in the folder which
312the report_device.php file was ran from. Here is an example of what
313these reports may be called:
314
315 MAH-H1-CNR01::file_show.html
316
317 MAH-H1-CNR01::get_bfd_session_information.html
318
319 MAH-H1-CNR01::get_bgp_summary_information.html
320
321 MAH-H1-CNR01::get_interface_information.html
322
323 MAH-H1-CNR01::get_msdp_peer_information.html
324
325 MAH-H1-CNR01::get_msdp_source_active_information.html
326
327 MAH-H1-CNR01::get_multicast_route_information.html
328
329 MAH-H1-CNR01::get_ospf_neighbor_information.html
330
331 MAH-H1-CNR01::get_pim_join_information.html
332
333
334Each file follows the naming format "Device Name :: Command ran" and
335the report is saved as an HTML file which can be viewed in any web
336browser. It is suggested that you do not have an index.html or
337index.php etc file within this folder so that you can use the web
338server index to browse these reports.
339
340- Viewing canned reports
341- Viewing results in webUI
342
343 Trouble Shoot::
344
345Segmentation fault, environment / bash variables
346
347- Where logs are stored and what they mean
348- Potential dependencies conflicts
349
350Unable to create directory `tmp': File exists at install-prereqs.pl
351line 608.
352-rm -rf tmp