· 7 years ago · Oct 21, 2018, 03:46 AM
1Ex. No: 1 STUDY OF ALL COMPUTING TECHNOLOGY
2Date:
3
4Aim:
5 To study the different computing technologies like Distributed Computing, Ubiquitous computing, Cluster computing and Mobile computing.
6Client server computing:
7 Client/Server computing is a computing model in which client and server computers communicate with each other over a network. In client/server computing, a server takes requests from client computers and shares its resources, applications and/or data with one or more client computers on the network, and a client is a computing device that initiates contact with a server in order to make use of a shareable resource.
8
9Peer to Peer Computing:
10 Peer-to-peer (P2P) is a decentralized communications model in which each party has the same capabilities and either party can initiate a communication session. Unlike the client/server model, in which the client makes a service request and the server fulfills the request, the P2P network model allows each node to function as both a client and server.
11
12
13Centralized Computing:
14 Centralized computing is computing done at a central location, using terminals that are attached to a central computer. The computer itself may control all the peripherals directly (if they are physically connected to the central computer), or they may be attached via a terminal server. Alternatively, if the terminals have the capability, they may be able to connect to the central computer over the network. The terminals may be text terminals or thin clients.
15
16Parallel Computing:
17 Parallel computing is a type of computing architecture in which several processors execute or process an application or computation simultaneously. Parallel computing helps in performing large computations by dividing the workload between more than one processor, all of which work through the computation at the same time. Most supercomputers employ parallel computing principles to operate.Parallel computing is also known as parallel processing.
18
19
20Distributed Computing:
21 Distributed computing is a computing concept that, in its most general sense, refers to multiple computer systems working on a single problem. In distributed computing, a single problem is divided into many parts, and each part is solved by different computers. As long as the computers are networked, they can communicate with each other to solve the problem. If done properly, the computers perform like a single entity.
22
23Grid Computing:
24 Grid computing is a processor architecture that combines computer resources from various domains to reach a main objective. In grid computing, the computers on the network can work on a task together, thus functioning as a supercomputer.
25
26Utility Computing:
27 Utility computing is the process of providing computing service through an on-demand, pay-per-use billing method. Utility computing is a computing business model in which the provider owns, operates and manages the computing infrastructure and resources, and the subscriber’s accesses it as and when required on a rental or metered basis.
28Pervasive Computing (or) ubiquitous computing:
29 Pervasive computing is an emerging trend associated with embedding microprocessors in day-to-day objects, allowing them to communicate information. It is also known as ubiquitous computing. The terms ubiquitous and pervasive signify "existing everywhere." Pervasive computing systems are totally connected and consistently available. It is also known as ubiquitous computing.
30Cluster Computing:
31 Cluster Computing addresses the latest results in these fields that support High Performance Distributed Computing (HPDC). In HPDC environments, parallel and/or distributed computing techniques are applied to the solution of computationally intensive applications across networks of computers. The journal represents an important source of information for the growing number of researchers, developers and users of HPDC environments.
32Fog Computing:
33 Fog computing or fog networking, also known as fogging, it is an architecture that uses edge devices to carry out a substantial amount of computation, storage, communication locally and routed over the internet backbone, and most definitively has input and output from the physical world known as transduction. Fog computing consists of Edge nodes directly performing physical input and output often to achieve sensor input, display output, or full closed loop process control, and may also use smaller Edge Clouds often called as Cloudlets at the Edge or nearer to the Edge than centralized Clouds residing in very large data centres. The processing power in advanced Edge Clouds like those that control autonomous vehicles can be considerable compared to more traditional Edge personal devices such as mobile phones and personal computers.
34Edge Computing:
35 "Edge computing" is used as a kind of catch-all for various networking technologies including peer-to-peer networking or ad hoc networking, as well as various types of cloud setups and other distributed systems. One other predominant type of edge networking is mobile edge networking or computing, an architecture that utilizes the edge of the cellular network for operations.
36Cloud Computing:
37 Cloud computing is a general term for the delivery of hosted services over the internet. Cloud computing enables companies to consume a compute resource, such as a virtual machine , storage or an application, as a utility -- just like electricity -- rather than having to build and maintain computing infrastructures in house.
38
39
40Ambient Computing:
41 Ambient computing is the backdrop of sensors, devices, intelligence, and agents that can put the Internet of Things to work. The Internet of Things (IoT) is maturing from its awkward adolescent phase.
42Internet Of Things (IOT):
43 The Internet of Things (IoT) is the network of physical devices, vehicles, home appliances, and other items embedded with electronics, software, sensors, actuators, and connectivity which enables these things to connect and exchange data, creating opportunities for more direct integration of the physical world into computer-based systems, resulting in efficiency improvements, economic benefits, and reduced human exertions. The number of IoT devices increased 31% year-over-year to 8.4 billion in 2017 and it is estimated that there will be 30 billion devices by 2020. The global market value of IoT is projected to reach $7.1 trillion by 2020.
44 IoT involves extending internet connectivity beyond standard devices, such as desktops, laptops, smart phones and tablets, to any range of traditionally dumb or non-internet-enabled physical devices and everyday objects. Embedded with technology, these devices can communicate and interact over the internet, and they can be remotely monitored and controlled.
45Data Analytics:
46 Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making. Data analysis has multiple facets and approaches, encompassing diverse techniques under a variety of names, while being used in different business, science, and social science domains.
47Data Mining:
48 Data mining is the process of discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems
49Data Warehouse:
50 In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence. DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place that are used for creating analytical reports for workers throughout the enterprise.
51Web Services and Service Architecture:
52 A service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Some means of connecting services to each other is needed.
53
54
55
56
57 Service-oriented architectures are not a new thing. The first service-oriented architecture for many people in the past was with the use DCOM or Object Request Brokers (ORBs) based on the CORBA specification. For more on DCOM and CORBA, see Prior Service-Oriented Architectures
58Result:
59 Thus the study experiment for different computing technologies was learned successfully.
60
61
62
63
64
65
66
67Ex. No: 2 STUDY OF GRID & CLOUD COMPUTING
68Date:
69
70Aim:
71 To study about grid and cloud computing.
72.Grid Computing:
73 Grid computing is the collection of computer resources from multiple places to reach a common goal. The grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application.
74 Grid computers also tend to be more heterogeneous and geographically dispersed (thus not physically coupled) than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries.
75
76Grid Architecture Layer:
77Grid computing is a distributed architecture of large numbers of computers connected to solve a complex problem. In the grid computing model, servers or personal computers run independent tasks and are loosely linked by the Internet or low-speed networks. Computers may connect directly or via scheduling systems.
78
79
80
81OGSA&OGSI Architecture:
82
83Cloud Computing:
84 Cloud computing is an information technology (IT) paradigm that enables ubiquitous access to shared pools of configurable system resources and higher-level services that can be rapidly provisioned with minimal management effort, often over the Internet. Cloud computing relies on sharing of resources to achieve coherence and economies of scale, similar to a public utility.
85
86Types of Cloud Computing:
87Based on a cloud location, we can classify cloud as:
88• public,
89• private,
90• hybrid
91• community cloud
92Public cloud:
93The public cloud is defined as computing services offered by third-party providers over the public Internet, making them available to anyone who wants to use or purchase them. They may be free or sold on-demand, allowing customers to pay only per usage for the CPU cycles, storage or bandwidth they consume.
94
95PRIVATE CLOUD:
96A private cloud is a cloud computing hardware and software platform that is dedicated to your organization. Private clouds hosted at Cybercon data center provides you the freedom to choose: network routers and switches, firewalls, server hardware, storage systems, and cloud computing software. Our solutions are designed and built to give you the power to construct and manage clouds across your internal data centers and Cybercon data centre on terms that you control. This means that you can keep a handle on compliance, security, and costs. And you can let your business needs drive your IT strategy, instead of having IT limit your options.
97
98HYBRID CLOUD:
99 Hybrid cloud is a cloud computing environment that uses a mix of on-premises, private cloud and third-party, public cloud services with orchestration between the two platforms. By allowing workloads to move between private and public clouds as computing needs and costs change, hybrid cloud gives businesses greater flexibility and more data deployment options.
100
101COMMUNITY CLOUD:
102 A community cloud is a cloud service model that provides a cloud computing solution to a limited number of individuals or organizations that is governed, managed and secured commonly by all the participating organizations or a third party managed service provider.
103
104Based on a service that the cloud is offering,
105• IaaS (Infrastructure-as-a-Service)
106• PaaS (Platform-as-a-Service)
107• SaaS (Software-as-a-Service)
108• or, Storage, Database, Information, Process, Application, Integration, Security, Management, Testing-as-a-service
109Infrastructure as a service:
110 Infrastructure as a service (IaaS) refers to online services that provide high-level APIs used to dereference various low-level details of underlying network infrastructure like physical computing resources, location, data partitioning, scaling, security, backup etc. A hypervisor, such as Xen, Oracle VirtualBox, Oracle VM, KVM, VMware ESX/ESXi, or Hyper-V, LXD, runs the virtual machines as guests..
111
112
113Platform-as-a-Service:
114Platform as a Service (PaaS) or application platform as a Service (aPaaS) or platform base service is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app. PaaS can be delivered in three ways:
115• As a public cloud service from a provider, where the consumer controls software deployment with minimal configuration options, and the provider provides the networks, servers, storage, operating_system (OS), middleware (e.g. Java runtime, .NET runtime, integration, etc.), database and other services to host the consumer's application.
116• As a private service (software or appliance) behind a firewall.
117• As software deployed on a public infrastructure as a service.
118
119Software-as-a-Service:
120Software as a service (SaaS /sæs/) is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted. It is sometimes referred to as "on-demand software", and was formerly referred to as "software plus services" by Microsoft. SaaS is typically accessed by users using a thin client via a web browser. SaaS has become a common delivery model for many business applications, including office software, messaging software, payroll processing software, DBMS software, management software, CAD_software,_development_software, gamification, virtualization, accounting, collaboration, customer_relationship_management (CRM).
121
122Merits:
123• Flexibility
124• Efficiency
125• Strategic value
126• Cost savings
127• Manageability
128Demerits:
129• Downtime
130• Security
131• Vendor Lock-In
132• Limited Control
133
134Result:
135 Thus the study experiment for grid and cloud computing was learned successfully.
136
137
138
139Ex. No: 3 EXECUTION OF LINUX COMMANDS IN UBUNTU
140Date:
141
142AIM:
143 To study the linux commands and execute the linux commands in Ubuntu.
144LINUX COMMANDS:
145 Linux is one of popular version of UNIX operating System. It is open source as its source code is freely available. It is free to use. Linux was designed considering UNIX compatibility. Its functionality list is quite similar to that of UNIX.
146 Components of Linux System:
147Linux Operating System has primarily three components
148• Kernel − Kernel is the core part of Linux. It is responsible for all major activities of this operating system. It consists of various modules and it interacts directly with the underlying hardware. Kernel provides the required abstraction to hide low level hardware details to system or application programs.
149• System Library − System libraries are special functions or programs using which application programs or system utilities accesses Kernel's features. These libraries implement most of the functionalities of the operating system and do not requires kernel module's code access rights.
150• System Utility − System Utility programs are responsible to do specialized, individual level tasks.
151 Kernel Mode vs User Mode:
152 Kernel component code executes in a special privileged mode called kernel mode with full access to all resources of the computer. This code represents a single process, executes in single address space and do not require any context switch and hence is very efficient and fast. Kernel runs each processes and provides system services to processes, provides protected access to hardware to processes.
153Basic Commands:
1541. Date Command:
155 This command is used to display the current data and time.
156 Syntax:
157 $date
1582. Calender Command:
159 This command is used to display the calendar of the year or the particular month of calendar year.
160Syntax:
161 a. $cal <year>
162 b. $cal <month> <year>
163 Here the first syntax gives the entire calendar for given year & the second Syntax gives the calendar of reserved month of that year.
1643. Echo Command:
165 This command is used to print the arguments on the screen.
166 Syntax:
167 $echo <text>
1684. Banner Command:
169 It is used to display the text in ‘#’ symbol .It displays the text in the form of a banner.
170Syntax :
171 $banner <arguments>
1725.’who’ Command:
173 It is used to display who are the users connected to our computer currently.
174 Syntax:
175 $who
1766.’who am i’ Command:
177 Display the details of the current working directory.
178 Syntax:
179 $who am i
1807.’tty’ Command:
181 It will display the terminal name.
182 Syntax:
183 $tty
1848.’CLEAR’ Command:
185It is used to clear the screen.
186 Syntax:
187 $clear
1889.’MAN’ Command:
189It helps us to know about the particular command and its options & working. It is like ‘help’ command in windows.
190 Syntax:
191 $man <command name>
19210. LIST Command:
193 It is used to list all the contents in the current working directory.
194 Syntax:
195 $ ls – options <arguments>
196If the command does not contain any argument means it is working in the Current directory.
197 Options:
198 a– used to list all the files including the hidden files.
199 c– list all the files columnwise.
200 d- list all the directories.
201 m- list the files separated by commas.
202 p- list files include ‘/’ to all the directories.
203 r- list the files in reverse alphabetical order.
204 f- list the files based on the list modification date.
205 x-list in column wise sorted order.
206DIRECTORY RELATED COMMANDS:
2071. Present Working Directory Command:
208 To print the complete path of the current working directory.
209 Syntax:
210 $pwd
2112. MKDIR Command:
212 To create or make a new directory in a current directory.
213 Syntax:
214 $mkdir <directory name>
2153. CD Command:
216 To change or move the directory to the mentioned directory .
217 Syntax: $cd <directory name>.
2184. RMDIR Command:
219 To remove a directory in the current directory & not the current directory itself.
220 Syntax:
221 $rmdir <directory name>
222FILE RELATED COMMANDS:
2231. CREATE A FILE:
224 To create a new file in the current directory we use CAT command.
225Syntax:
226 $cat > filename.
2272. DISPLAY A FILE:
228 To display the content of file mentioned we use CAT command without ‘>’ operator.
229 Syntax:
230 $cat filename.
2313. COPYING CONTENTS:
232 To copy the content of one file with another. If file doesnot exist, a new file is created and if the file exists with some data then it is overwritten.
233 Syntax :
234 $ cat <source filename> >> <destination filename>
2354. SORTING A FILE:
236 To sort the contents in alphabetical order in reverse order.
237 Syntax:
238 $sort <filename >
239 Option: $ sort –r <filename>
2405. COPYING CONTENTS FROM ONE FILE TO ANOTHER:
241 To copy the contents from source to destination file so that both contents are same.
242 Syntax: $cp <source filename> <destination filename>
243 $cp <source filename path > <destination filename path>
2446. MOVE Command:
245 To completely move the contents from source file to destination file and to remove the source file.
246 Syntax:
247 $ mv <source filename> <destination filename>
2487. REMOVE Command:
249 To permanently remove the file we use this command.
250 Syntax:
251 $rm <filename>
2528. WORD Command:
253 To list the content count of no of lines, words, characters.
254 Syntax:
255 $wc<filename>
256 Options:
257 -c – to display no of characters.
258 -l – to display only the lines.
259 -w – to display the no of words.
260
2619. LINE PRINTER:
262 To print the line through the printer, we use lp command.
263 Syntax: $lp <filename>
26410. PAGE Command:
265 This command is used to display the contents of the file page wise & next page can be viewed by pressing the enter key.
266 Syntax:
267 $pg <filename>
26811. FILTERS AND PIPES
269HEAD: It is used to display the top ten lines of file.
270 Syntax: $head<filename>
271
272TAIL: This command is used to display the last ten lines of file.
273 Syntax: $tail<filename>
274PAGE: This command shows the page by page a screenfull of information is displayed after which the page command displays a prompt and passes for the user to strike the enter key to continue scrolling.
275 Syntax: $ls –a\p
276MORE: It also displays the file page by page .To continue scrolling with more command, press the space bar key.
277 Syntax: $more<filename>
278GREP: This command is used to search and print the specified patterns from the file. Syntax: $grep [option] pattern <filename>
279SORT: This command is used to sort the datas in some order.
280 Syntax: $sort<filename>
281PIPE: It is a mechanism by which the output of one command can be channeled into the input of another command.
282 Syntax: $who | wc-l
283TR: The tr filter is used to translate one set of characters from the standard inputs to another.
284 Syntax: $tr “[a-z]†“[A-Z]â€
285COMMUNICATION THROUGH UNIX COMMANDS
286Command: MESG
287 Description: The message command is used to give permission to other users to send message to your terminal.
288 Syntax: $mesg y
289Command: WRITE
290Description: This command is used to communicate with other users, who are logged in at the same time.
291Syntax: $write <user name>
292Command: WALL
293Description: This command sends message to all users those who are logged in using the unix server.
294Syntax: $wall <message>
295Command: MAIL
296Description: It refers to textual information, that can be transferred from one user to another
297Syntax: $mail <user name>
298 Command: REPLY
299 Description: It is used to send reply to specified user.
300 Syntax: $reply<user name>
301
302
303vi EDITOR COMMANDS
304The vi editor is a visual editor used to create and edit text, files, documents and programs. It displays the content of files on the screen and allows a user to add, delete or change part of text. There are three modes available in the vi editor, they are
305 1. Command mode
306 2. Input (or) insert mode.
307The vi editor is invoked by giving the following commands in UNIX prompt.
308 Syntax: $vi <filename> (or)$vi
309 Options :
310 1.vi +n <filename> - this would point at the nth line (cursor pos).
311 2.vi –n <filename> - This command is to make the file to read only to change from one mode to another press escape key.
312INSERTING AND REPLACING COMMANDS:
313To move editor from command node to edit mode, you have to press the <ESC> key. For inserting and replacing the following commands are used.
3141. ESC a Command:
315This command is used to move the edit mode and start to append after the current character.
316 Syntax : <ESC> a
3172. ESC A COMMAND :
318This command is also used to append the file , but this command append at the end of current line.
319 Syntax: <ESC> A
3203. ESC i Command:
321 This command is used to insert the text before the current cursor position.
322 Syntax: <ESC> i
3234. ESC I Command:
324 This command is used to insert at the beginning of the current line.
325 Syntax: <ESC> I
3265. ESC o Command:
327This command is insert a blank line below the current line & allow insertion of contents.
328 Syntax: <ESC> o
3296. ESC O Command:
330This command is used to insert a blank line above & allow insertion of contents.
331 Syntax : <ESC> O
3327. ESC r Command :
333 This command is to replace the particular character with the given
334 characters.
335 Syntax: <ESC> rx Where x is the new character.
3368. ESC R Command:
337 This command is used to replace the particular text with a given text.
338Syntax: <ESC> R text
3399. <ESC> s Command:
340 This command replaces a single character with a group of character.
341 Syntax: <ESC> s
34210.<ESC> S Command :
343 This command is used to replace a current line with group of characters. Syntax : <ESC> S
344RESULT:
345 Thus the execution of linux commands in ubuntu environment was executed successfully.
346
347
348
349
350Ex. No: 4 Installation of C compiler in the virtual machine
351Date:
352Aim:
353To install a C compiler in the virtual machine and execute some sample program in cloud environment.
354PROCEDURE:
355STEP 1: Click oracle VM Virtual Box (opened)
356STEP 2: Click File [Oracle Virtual Box Manager]
357STEP 3: Click Import Appliance
358Appliance to Import
359
360Click browser (go to D: or E :)
361STEP 4: In D: or E:, Click Hadoop & open nebula folder
362STEP 5: In that, click ubuntu-grid (open)
363STEP 6: Click next
364STEP 7: Click import
365STEP 8:
366Import virtual disk image
367(It will take time to import)
368STEP 9: Click ubuntu *(click start)
369STEP 10: Click ok
370STEP 11:
371File machine view input devices help
372User name:
373Dinesh
374Password:
37599425
376
377STEP 12:
378
379 Click here and type terminal
380Or
381Directly click terminal
382
383STEP 13: To open a new file..Syntax: $vi filename.c
384STEP 14:
385~
386~
387Go to insert mode
388Click esc+i
389STEP 15: After typing C program press esc+:wq(to save & quit)
390STEP 16: cc Filename.c (to compile)
391STEP 17:./a.out(to run)
392
393Program 1: Switch case
394#include<stdio.h>
395void main()
396{
397int a,b,ch;
398int c;
399printf("Enter two numbers\n");
400scanf("%d%d",&a,&b);
401do
402{
403printf("\nEnter your choice\n");
404scanf("%d",&ch);
405switch(ch)
406{
407case 1:
408printf("Addition is %d",a+b);
409break;
410case 2:
411printf("Subtraction is %d",a-b);
412break;
413case 3:
414printf("Multiplication is %d",a*b);
415break;
416case 4:
417printf ("Division is %d", a/b);
418break;
419default:
420printf ("Enter valid choice");
421break;
422}
423printf("Do you want to continue y/n");
424scanf("%d",&c);
425}while(c==0);
426}
427output:
428Enter two numbers
4293
4302
431Enter your choice
4321
433Addition is 5
434Do you want to continue 0/1
4350
436Enter your choice
4375
438Enter valid choice
439Do you want to continue 0/1
4401
441
442
443program2 : Armstrong Number
444#include<stdio.h>
445void main()
446{
447int n,a=0,r=0,rf=0;
448printf("Enter the number\n");
449scanf("%d",&n);
450a=n;
451while(n!=0)
452{
453r=n%10;
454rf+=r*r*r;
455n=n/10;
456}
457if(rf==a)
458printf("entered number is armstrong number");
459else
460printf("entered number is not an armstrong number");
461}
462Output:
463Enter the number
464153
465entered number is armstrong number
466
467
468program 3: Factorial
469#include<stdio.h>
470void main()
471{
472int n,f=1,i;
473printf("Enter the number\n");
474scanf("%d",&n);
475for(i=1;i<=n;i++)
476f=f*i;
477printf("factorial of %d is %d",n,f);}
478output:
479Enter the number:4
480factorial of 4 is 24
481
482
483
484
485
486Program 4: Addition of two numbers
487include<stdio.h>
488void main()
489{
490int a,b,c;
491printf("enter two numbers");
492scanf("%d%d",&a,&b);
493c=a+b;
494printf("Sum of a and b is %d",c);
495}
496
497Output:
498Enter two numbers
49923
50027
501Sum of a and b is 50
502
503
504Program 5: Palindrome
505#include<stdio.h>
506void main()
507{
508printf("Enter the number");
509
510int n,a=0,r=0,rf=0;
511scanf("%d",&n);
512a=n;
513while(n!=0)
514{
515r=n%10;
516rf=rf*10+r;
517n=n/10;
518}
519if(rf==a)
520printf("Entered number is palindrome");
521else
522printf("Entred number is not palindrome");
523}
524
525Output:
526Enter the number
52733
528Entered number is palindrome
529
530
531Program 6: Multiplication Table
532#include<stdio.h>
533void main()
534{
535int m,n,i;
536printf("Enter which table you want");
537scanf("%d",&m);
538printf("Enter the range of the table");
539scanf("%d",&n);
540for(i=1;i<=n;i++)
541printf("%d x %d = %d \n",i,m,i*m);
542}
543Output:
544Enter which table you want
5452
546Enter the range of the table
5473
5481 x 2 = 2
5492 x 2 = 4
5503 x 2 = 6
551 Result:
552 Thus C compiler is installed in virtual machine and some sample programs are executed and verified in cloud environment.
553
554
555
556
557
558Ex. No: 5 RUNNING VM’S OF DIFFERENT CONFIGURATION
559 IN OPEN NEBULA
560Date:
561AIM:
562To find procedure to run the virtual machine of different configuration and to check how
563 many virtual machines can be utilized at particular time.
564PROCEDURE:
565Install Open nebula sandbox:
5661. Open Virtual box
5672. File ïƒ import Appliance
5683. Browse OpenNebula-Sandbox-5.0.ova file
5694. Then go to setting, select Usb and choose USB 1.1
5705. Then Start the Open Nebula
5716. Login using username: root, password:opennebula
572
573
574
575Procedure to run the virtual machine of different configuration and to check how many
576virtual machines can be utilized at particular time:
5771. Open Browser, type localhost:9869
5782. Login using username: oneadmin, password: opennebula
5793. Click on instances, select VMs then follow the steps to create Virtaul machine
580a. Expand the + symbol
581b. Select user oneadmin
582c. Then enter the VM name,no.of instance, cpu.
583d. Then click on create button.
584e. Repeat the steps the C,D for creating more than one VMs.
585
586RESULT:
587Thus to find procedure to run the virtual machine of different configuration and to check how many virtual machines can be utilized at particular time using open nebula is done and verified.
588
589Ex. No: 6 PROCEDURE TO ATTACH VIRTUAL BLOCK TO THE VM
590Date:
591AIM:
592 To find procedure to attach virtual block to the virtual machine and to check whether it holds the data even after the release of the virtual machine.
593PROCEDURE:
594 Method 1:
5951. Open the virtual box
5962. Power off the VM which you want to add virtual box
5973. Then right click on that VM,select setting
5984. Then click on storage,find controller IDE .
5995. In the top right find add hard disk icon, the pop up window display
6006. On that window select create new disk, and then click next and next then finish.
6017. Then find attributes icon ,hard disk as IDE secondary slave.
602
603Method 2:
6041. Open Browser, type localhost:9869
6052. Login using username: oneadmin, password: opennebula
6063. Click on instances, select VMs then follow the steps to add virtual block
607a. Select any one VM from the list and power off the VM
608b. Then click on that VM ,find the storage tab then click on that
609c. Then find the attach disk button
610d. Click on that button ,the new pop window display
611e. On that window select either image or volatile disk
612f. Click on attach butto
613RESULT:
614 Thus to find procedure to attach virtual block to the virtual machine and to check whether it holds the data even after the release of the virtual machine is executed and verified.
615
616
617
618
619
620Ex. No: 7 The Virtual Machine Migration
621Date:
622
623AIM:
624Learning the procedure to migrate the virtual machine from one host to another, perform the virtual machine migration and show the virtual machine migration based on certain condition from one node to the other.
625PROCEDURE:
6261. Open Browser, type localhost:9869
6272. Login using username: oneadmin, password: opennebula
6283. Then follow the steps to migrate VMs
629a. Click on infrastructure
630b. Select clusters and enter the cluster name
631c. Then select host tab, and select all host
632d. Then select Vnets tab, and select all vnet
633e. Then select data stores tab, and select all data stores
634f. And then choose host under infrastructure tab
635g. Click on + symbol to add new host, name the host then click on create.
6364. on instances, select VMs to migrate then follow the steps
637a. Click on 8th icon ,the drop down list display
638b. Select migrate on that ,the popup window display
639c. On that select the target host to migrate then click on migrates.
640Before migration
641 Host: naveenkumar
642Host:one-sa
643After Migration:
644 Host:one-sandbox
645Host:naveenkumar
646RESULT:
647 The procedure of creating the host with an existing image file and migrating the virtual machine is practiced and recorded through this experiment.
648
649
650
651
652
653
654
655
656Ex.No: 8 Develop a new Web Service for Calculator
657Date:
658Aim:
659To develop a web service program for a calculator.
660Procedure:
661Step1. Open netbeans and go to New
662
663
664
665
666
667
668
669
670
671
672
673
674
675Step 2. Choose Java Web and select Web application and give next.
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690Step 3. Enter the project name and give next and Select the Server either tomcat or glas
691
692
693
694
695
696
697
698
699
700Step 4. Give next and select finish
701Step 5. Right click the WebApplication (Projec Name) and Select New, and choose Java Class
702
703
704
705
706
707
708
709
710
711Step 6. Type the following code
712import javax.jws.WebMethod;
713import javax.jws.WebParam;
714import javax.jws.WebService;
715
716@WebService(serviceName="MathService", targetNamespace = "http://my.org/ns/") public class MathService {
717@WebMethod(operationName = "hello"
718public String hello(@WebParam(name="name")String txt){ return "Hello"+txt+"!";
719}
720@WebMethod(operationName = "addSer"
721public String addSer(@WebParam(name="value1")int v1, @WebParam(name = "value2")int v2){
722return "Answer:" +(v1+v2)+"!";
723}
724@WebMethod(operationName = "subSer")
725public String subSer(@WebParam(name="value1")int v1, @WebParam(name = "value2")int v2){
726return "Answer:" +(v1-v2)+"!";
727}
728@WebMethod(operationName = "mulSer")
729public String mulSer(@WebParam(name="value1")int v1, @WebParam(name = "value2")int v2){
730return "Answer:" +(v1*v2)+"!";
731}
732@WebMethod(operationName = "divSer")
733public String divSer(@WebParam(name="value1")int v1, @WebParam(name = "value2")int v2){float res= 0; try
734{res = ((float)v1)/((float) v2); return "Answer:" +res+"!";
735}
736catch(ArithmeticException e){ System.out.println("Can't be divided by Zero"+e); return "Answer:" +e.getMessage().toString()+"!!!";
737}
738}
739}
740Step 7. Run Project by pressing F6 key or Run button.
741Step 8. Check Web browser
742for the following name is available else give it http://localhost:8080/Web pplication2/Math ervice?Tester
743MathService?Tester ---> represents the java class name
744
745
746
747
748
749
750
751
752
753
754
755 Output Screen:
756Give some value in the fields and check the out put by pressing enter key.
757 Finally select the WSDL link
758
759Result:
760 Thus the program on calculator for web services is executed successfully.
761
762
763
764
765
766
767
768Ex.No: 9 Implementation Of OGSA using windows
769Date:
770
771
772Aim:
773 To develop a new OGSA- Compliant web service using windows.
774
775Procedure:
776I . Setup the Development Environment
777
7781.1. First you need to set up the development environment. Following things are needed if you want to create Web Services using Axis2 and Eclipse IDE.
779
780Some Eclipse versions have compatibility issues with Axis2. This tutorial is tested with Apache Axis2 1.5.2, Eclipse Helios and Apache Tomcat 6.
781
7821) Apache Axis2 Binary Distribution - Download
7832) Apache Axis2 WAR Distribution - Download
7843) Apache Tomcat - Download
7854) Eclipse IDE – Download
7865) Java installed in your Computer – Download
787
7881.2. Then you have to set the environment variables for Java and Tomcat. There following variables should be added.
7891
7902
7913 JAVA_HOME :- Set the value to jdk directory (e.g. C:\Program Files\Java\jdk1.6.0_21)
792TOMCAT_HOME :- Set the value to top level directory of your Tomcat install (e.g. D:\programs\apache-tomcat-6.0.29)
793PATH :- Set the value to bin directory of your jdk (e.g. C:\Program Files\Java\jdk1.6.0_21\bin)
7941.3. Now you have to add runtime environment to eclipse. There go to Windows –-> Preferences and Select the Server --> Runtime Environments.
795
796
797
798There select Apache Tomcat v6.0 and in the next window browse your Apache installation directory and click finish.
799
800
801
8021.4. Then click on the Web Service –-> Axis2 Preferences and browse the top level directory of Apache Axis2 Binary Distribution.
803
804
805
806II. Creating the Web Service Using Bottom-Up Approach
807
8082.1 First create a new Dynamic Web Project (File --> New –-> Other…) and choose Web --> Dynamic Web Project.
809
810
811
8122.2 Set Apache Tomcat as the Target Runtime and click Modify to install Axis2 Web Services project facet.
813
814
815
8162.3 Select Axis2 Web Services
817
818
819
8202.4 Click OK and then Next. There you can choose folders and click Finish when you are done.
821
822III. Create Web Service Class
823
824Now you can create a Java class that you would want to expose as a Web Service. I’m going to create new class called FirstWebService and create public method called addTwoNumbers which takes two integers as input and return the addition of them.
825
8263.1 Right Click on MyFirstWebService in Project Explorer and select New –-> Class and give suitable package name and class name. I have given com.sencide as package name and FirstWebService as class name.
827
828
829package com.sencide;
830public class FirstWebService {
831 public int addTwoNumbers(int firstNumber, int secondNumber){
832 return firstNumber + secondNumber;
833 }
834}
8353.2 Then, select File --> New –-> Other and choose Web Service.
836
837
838
8393.3 Select the FirstWebService class as service implementation and to make sure that the Configuration is setup correctly click on Server runtime.
840
841
842
8433.4 There set the Web Service runtime as Axis2 (Default one is Axis) and click Ok.
844
845
846
8473.5 Click Next and make sure Generate a default service.xml file is selected.
848
849
850
8513.6 Click Next and Start the Server and after server is started you can Finish if you do not want to publish the Web service to a test UDDI repository.
852
853
854
855You can go to http://localhost:8888/MyFirstWebService/services/listServices to see your running service which is deployed by Axis2. You can see the WSDL by clicking the link FirstWebService.
856
857
858Result:
859 Thus a new OGSA- Compliant web service is developed and verified using windows .
860
861
862
863
864
865
866
867Ex.No:10 To develop a new OGSA complaint Webservice.(Ubuntu-grid)
868Date:
869Aim:
870 To develop a new OGSA complaint Webservice
871Procedure:
872Step 1: Choose New Project from the main menu
873
874
875
876
877
878
879
880
881Step 2: Select POM project from the maven category
882
883
884
885
886
887
888
889Step 3: Type MavenOSGiCDIProject as the project name and click finish. When you click finish, the IDE creates the POM project and opens the project in the project window.
890
891
892
893
894
895
896
897Step 4: Expand the project files node in the project window and double – click pom.xml to open the file in editor and do the modification in the file and save.
898In pom.xml file
899<?xml version="1.0" encoding="UTF-8"?>
900<project xmlns="http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XML chema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.mycompany</groupId>
901<artifactId>MavenO GiCDIProject</artifactId> <version>1.0-SN PSHOT</version> <packaging>pom</packaging><properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencyManagement><dependencies><dependency> <groupId>org.osgi</groupId>
902<artifactId>org.osgi.core</artifactId>
903<version>4.2.0</version>
904<scope>provided</scope>
905</dependency></dependencies></dependencyManagement>
906</project>
907Step 5:Creating OGSi Bundle Projects
908Choose File -> New Project to open the New Project Wizzard
909Step 6 : Choose OGSI Bundle from Maven category. Click Next
910
911
912
913
914
915
916
917
918Step 7: Creating MavenHelloService pi as the Project Name for OGSi Bundle
919
920
921
922
923
924
925
926
927
928The IDE creates the bundle project and opens the project in the Project Window. And check thebuilding pugins at pom.xml under project files.
929
930
931
932
933
934
935
936As well as it will create org.osgi.core artifact as default and it can be view at under Dependencies.
937Step 7: Buid the MavenHello ervice pi Project by
938
9391. Right Click the MavenHello ervice pi project node in the project window and choose properties.
940
941
942
943
944
945
9462. Select the source category in the project project dialog box
9473. Set the Source/Binary Format to 1.6 and confirm that the Encoding is UTF-8 and click ok
948
949
950
951
952
953
9544. Right click the source package node in the project window and choose New -> JavaInterface
9555. Type Hello for the Class Name
956
957
958
959
960
961
962
963
964
9656. Select com.mycompany.mavenhelloserviceapi as the Package. Click finish.
9667. Add the following sayHello method to the interface and save the changes.
967package com.mycompany.mavenhelloserviceapi;
968public interface Hello {
969 String sayHello(String name);
970}
971
972
973
974
975
976
977
978
979
980
981
982
983
984
9858. Right click the project node in the project window and choose build.
986
9879. After building the project, open files window and expand the project node such that you can see MavenHelloServiceApi-1.0-SN PSHOT.jar is created in the target folder.
988
989
990
991
992
993
994
995Step 8: Creating the MavenHelloServiceImpl Implementation Bundle
996Here you will create the MavenHelloServiceImpl in the POM Project.
9971. Choose File -> New Project to open the New Project Wizard
9982. Choose OSGi Bundle from the Maven category. Click Next.
9993. Type MavenHelloServiceImpl for the Project Name
10004. Click Browse and select the MavenOSGiCDIProject POM project as the Location. Click Finish.(As earlier step).
10015. Right click the project node in the Projects window and choose Properties.
10026. Select the Sources category in the Project Properties dialog box.
10037. Set the Source/Binary Format to 1.6 and confirm that the Encoding is UTF-8. Click OK.
10048. Right click Source Packages node in the Projects window and choose New -> Java Class.
10059. Type HelloImpl for the Class Name.
100610. Select com.mycompany.mavenhelloserviceimpl as the Package. Click Finish.
100711. Type the following and save your changes
1008package com.mycompany.mavenhelloserviceimpl;
1009public class HelloImpl implements Hello { public String sayHello(String name){
1010return "Hello" +name;
1011}}
1012 When you implement Hello, the IDE will display an error that you need to resolve by adding the MavenHelloServiceApi project as a dependency.
101312. Right click the Dependencies folder of MavenHelloServiceImpl in the Projects window and choose Add Dependency.
101413. Click the Open Projects tab in the dd Library dialog. 14. Select MavenHello ervice pi O Gi Bundle. Click dd
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
102614. Expand the com.mycompany.mavenhelloserviceimpl package and double click Activator.java and open the file in editor.
1027
1028The IDE automatically creates the Activator.java bundle and its manage the lifecycle of bundle. By default it includes start() and stop().Modify the start() and Stop() methods in the bundle activator class by adding thefollowing lines.
1029package com.mycompany.mavenhelloserviceimpl;
1030import org.osgi.framework.BundleActivator;
1031import org.osgi.framework.BundleContext;
1032public class Activator implements BundleActivator {
1033public void start(BundleContext context) throws Exception { // TODO add activation code here
1034System.out.println("HelloActivator::start"); context.registerService(Hello.class.getName(),new HelloImpl(),null); System.out.println("HelloActivator::registration of Hello Service Successfull");
1035}
1036public void stop(BundleContext context) throws Exception { // TODO add deactivation code here
1037context.ungetService(context.getServiceReference(Hello.class.get ame())); System.out.println("HelloActivator stopped");
1038}
1039}
1040
1041
1042
1043
1044
1045
1046
1047
1048Step 9: Bulding and Deploying the OSGi Bundles
1049Here you will build the OSGi bundles and deploy the bundles to GlassFish
10501. Right click the MavenOSGiCDIProject folder in the Projects window and choose Clean and Build.
1051## When you build the project the IDE will create the JAR files in the target folder and aslo install the snapshot JAR in the local repository.
1052## In file window, by expanding the target folder of each of the two bundle projects it will show two JAR archieves(MavenHelloServiceApi-1.0-SNAPSHOT.jar and MavenHElloServiceImpl-1.0-SNAPSHOT.jar.)
1053
1054
1055
1056
1057
1058
1059
10602. Start the GlassFish server (if not already started)
10613. Copy the MavenHelloService pi-1.0-SN PSHOT.jar to the /home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles ( GlassFish installed Directory)
10624. You can see output similar to the following in the GlassFish Server log in the output window.
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072Info: Installed /home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceApi-1.0-SNAPSHOT.jar
1073Info: Started bundle: file:/home/linux/glassfish-
10744.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceApi-1.0-SNAPSHOT.jar
1075Info: Started bundle: file:/home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceApi-1.0-SNAPSHOT.jar
10765. Repeat the step of copying the MavenHelloServiceImpl-1.0-SNAPSHOT.jar to the/home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles
1077( GlassFish installed Directory)
10786. You can see the output athe glassfish server log
1079
1080
1081
1082
1083
1084
1085
1086Info: Installed /home/linux/glassfish 4.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceImpl-1.0-SNAPSHOT.jarInfo: Hello ctivator::start
1087Info: Hello ctivator::registration of Hello ervice uccessfull Info: Started bundle: file:/home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceImpl-1.0-SN PSHOT.jar
1088PREPARATION 30
1089PERFORMANCE 30
1090RECORD 40
1091TOTAL 100
1092Info: Started bundle: file:/home/linux/glassfish-4.1.1/glassfish/domains/domain1/autodeploy/bundles/MavenHelloServiceImpl-1.0-SN PSHOT.jar
1093Result:
1094Thus a new OGSA- complaint web service has been executed successfully.
1095
1096
1097
1098
1099
1100
1101Ex.No:11 To develop a Grid Service using Apache Axis
1102Date:
1103
1104Aim:
1105 To develop a Grid Service using Apache Axis
1106Procedure:
1107Using Apache Axis develop a Grid Service.
11081. Open the terminal
11092. Type cd /opt/axis2/axis2-1.7.3/bin then press enter
11103. Type chmod 500 axis2server.sh
11114. Type ./axis2server.sh
11125. Then open browser on ubuntu type the URL as localhost:8080/axis2/services
1113
1114
1115Result:
1116Thus Grid Service using Apache Axis has been executed successfully.
1117
1118
1119
1120
1121
1122
1123
1124Ex.No:12 Develop applications using Java or C/C++ Grid APIs
1125Date:
1126Aim:
1127 To Develop applications using Java or C/C++ Grid APIs
1128Procedure:
1129Develop applications using Java or C/C++ Grid APIs
1130
1131a. Open the terminal
1132b. Type cd /opt/axis2/axis2-1.7.3/bin then press enter
1133c. gedit hello.c
1134d. gcc hello.c
1135e. ./a.out
1136Program 1: Switch case
1137#include<stdio.h>
1138void main()
1139{
1140int a,b,ch;
1141int c;
1142printf("Enter two numbers\n");
1143scanf("%d%d",&a,&b);
1144do
1145{
1146printf("\nEnter your choice\n");
1147scanf("%d",&ch);
1148switch(ch)
1149{
1150case 1:
1151printf("Addition is %d",a+b);
1152break;
1153case 2:
1154printf("Subtraction is %d",a-b);
1155break;
1156case 3:
1157printf("Multiplication is %d",a*b);
1158break;
1159case 4:
1160printf("Division is %d",a/b);
1161break;
1162default:
1163printf("Enter valid choice");
1164break;
1165}
1166printf("Do you want to continue y/n");
1167scanf("%d",&c);
1168}while(c==0);
1169}
1170
1171output:
1172Enter two numbers
11733
11742
1175Enter your choice
11761
1177Addition is 5
1178Do you want to continue 0/1
11790
1180Enter your choice
11815
1182Enter valid choice
1183Do you want to continue 0/1
11841
1185
1186
1187
1188
1189program2 : Armstrong Number
1190
1191#include<stdio.h>
1192void main()
1193{
1194int n,a=0,r=0,rf=0;
1195printf("Enter the number\n");
1196
1197scanf("%d",&n);
1198a=n;
1199while(n!=0)
1200{
1201r=n%10;
1202rf+=r*r*r;
1203n=n/10;
1204}
1205if(rf==a)
1206printf("entered number is armstrong number");
1207else
1208printf("entered number is not an armstrong number");
1209}
1210output:
1211Enter the number
1212153
1213entered number is armstrong number
1214 Result:
1215Thus applications using Java or C/C++ Grid APIs has been executed successfully.
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225Ex.No:13 Develop secured applications using basic security mechanisms
1226Date:
1227
1228Aim:
1229 To develop secured applications using basic security mechanisms available in Globus Toolkit
1230Procedure:
1231 Develop secured applications using basic security mechanisms available in Globus
1232Toolkit.
12331. Follow these command to install basic security
1234Installing GRID Essential
1235wget http://www.globus.org/ftppub/gt6/installers/repo/globus-toolkit-repo_latest_all.deb
1236sudo dpkg -i globus-toolkit-repo_latest_all.deb
1237 sudo apt-get update
1238 sudo apt-get install globus-data-management-client
1239 sudo apt-get install globus-gridftp
1240 sudo apt-get install globus-gram5
1241 sudo apt-get install globus-gsi
1242 sudo apt-get install globus-data-management-server
1243 sudo apt-get install globus-data-management-client
1244 sudo apt-get install globus-data-management-sdk
1245 sudo apt-get install globus-resource-management-server
1246 sudo apt-get install globus-resource-management-client
1247 sudo apt-get install globus-resource-management-sdk
1248 sudo apt-get install myproxy
1249 sudo apt-get install gsi-openssh
1250 sudo apt-get install globus-gridftp globus-gram5 globus-gsi myproxy myproxy-server myproxy-admin
12512. After installing like myproxy,gsi-openssh and Globus GRAM file
1252a. Click the fileïƒ computer
1253b. Then Search Grid security folder
1254PREPARATION 30
1255PERFORMANCE 30
1256RECORD 40
1257TOTAL 100
1258c. Then see the gsi.conf, sshftp.( This indicate the basic security mechanisms are configured)
1259
1260Result:
1261Thus secured applications using basic security mechanisms available in Globus Toolkit
1262 has been executed successfully.
1263
1264
1265
1266
1267
1268
1269
1270Ex. No: 14 Find procedure to set up the one node Hadoop cluster.
1271Date:
1272
1273AIM:
1274To Set up the one node Hadoop cluster.
1275
1276PRE-REQUISITE:
12771. Java v1.8 installation
12782. Configuring SSH access. sudo apt-get install vim
1279This exercise has been created for following environment:
1280• Ubuntu Linux 64-bit
1281• JDK 1.8.0_05
1282• Hadoop 2.7.x stable release
1283Note: This exercise depicts about only compatible versions of Hadoop ecosystem tools and software downloaded from the official Apache hadoop website. Preferably use a stable release of the particular tool.
1284
1285PROCEDURE:
12861) Installing Java
1287Hadoop is a framework written in Java for running applications on large clusters of commodity hardware. Hadoop needs Java 6 or above to work.
1288Step 1: Download tar and extract
1289Download Jdk tar.gz file for linux-62 bit, extract it into “/usr/localâ€
1290# cd /opt
1291# sudo tar zxvf /home/user/Downloads/jdk-8u5-linux-x64.tar.g
1292# cd /home/user/Downloads/jdk1.8.0_05
1293Step 2: Set Environments
1294• Open the “/etc/profile†file and Add the following line as per the version
1295• Set a environment for Java
1296• Use the root user to save the /etc/proflie or use gedit instead of vi .
1297• The 'profile' file contains commands that ought to be run for login shells
1298# sudo nano /etc/profile
1299#--insert JAVA_HOME JAVA_HOME=/home/user/Downloads/jdk1.8.0_05
1300#--in PATH variable just append at the end of the line
1301PATH=$PATH:$JAVA_HOME/bin
1302#--Append JAVA_HOME at end of the export statement export PATH JAVA_HOME
1303save the file using by pressing “Esc†key followed by :wq!
1304Step 3: Source the /etc/profile
1305# source /etc/profile
1306Step 4: Update the java alternatives
13071. By default OS will have a open jdk. Check by “java -versionâ€. You will be prompt
1308 “openJDKâ€
13092. If you also have openjdk installed then you'll need to update the java alternatives:
13103. If your system has more than one version of Java, configure which one your system causes by entering the following command in a terminal window
13114. By default OS will have a open jdk. Check by “java -versionâ€. You will be prompt
1312 “JavaHotSpot(TM) 64-Bit Serverâ€
1313# update-alternatives --install "/usr/bin/java" java /home/user/Downloads/jdk1.8.0_05/bin/java"
13141)# update-alternatives --config java
1315--type selection number:
1316# java -version
13172) configure ssh
1318• Hadoop requires SSH access to manage its nodes, i.e. remote machines plus your local machine if you want to use Hadoop on it (which is what we want to do in this exercise). For our single-node setup of Hadoop, we therefore need to configure SSH access to localhost
1319• The need to create a Password-less SSH Key generation based authentication is so that the master node can then login to slave nodes (and the secondary node) to start/stop them easily without any delays for authentication
1320• If you skip this step, then have to provide password
1321Generate an SSH key for the user. Then Enable password-less SSH access to yo sudo apt-get install openssh-server
1322--You will be asked to enter password, root@ubuntu # ssh localhost root@ubuntu# ssh-keygen root@ubuntu# ssh-copy-id -i localhost
1323--After above 2 steps, You will be connected without password, root@ubuntu# ssh localhost
1324root@ubuntu# exit
13253) Hadoop installation
1326• Now Download Hadoop from the official Apache, preferably a stable release version of
1327Hadoop 2.7.x and extract the contents of the Hadoop package to a location of your choice.
1328
1329• For example, choose location as “/opt/â€
1330Step 1: Download the tar.gz file of latest version Hadoop ( hadoop-2.7.x) from the official site .
1331Step 2: Extract (untar) the downloaded file from this commands to /opt/bigdata
1332 root@ubuntu# cd /opt
1333root@ubuntu# sudo tar zxvf /home/user/Downloads/hadoop-2.7.0.tar.g
1334root@ubuntucd hadoop-2.7.0/
1335Like java, update Hadop environment variable in /etc/profile
1336# sudo nano /etc/profile
1337#--insert HADOOP_PREFIX HADOOP_PREFIX=/home/user/Downloads/hadoop-2.7.0
1338#--in PATH variable just append at the end of the line
1339PATH=$PATH:$HADOOP_PREFIX/bin
1340#--Append HADOOP_PREFIX at end of the export statement export PATH JAVA_HOME HADOOP_PREFIX
1341save the file using by pressing “Esc†key followed by :wq!
1342Step 3: Source the /etc/profile
1343# source /etc/profile
1344Verify Hadoop installation
1345# cd $HADOOP_PREFIX
1346# bin/hadoop version
13473.1) Modify the Hadoop Configuration Files
1348• In this section, we will configure the directory where Hadoop will store its configuration files, the network ports it listens to, etc. Our setup will use Hadoop Distributed File System,(HDFS), even though we are using only a single local machine.
1349• Add the following properties in the various hadoop configuration files which is available under $HADOOP_PREFIX/etc/hadoop/
1350• core-site.xml, hdfs-site.xml, mapred-site.xml & yarn-site.xml
1351Update Java, hadoop path to the Hadoop environment file
1352# cd $HADOOP_PREFIX/etc/hadoop
1353# nano hadoop-env.sh
1354Paste following line at beginning of the fill
1355export JAVA_HOME=/usr/local/jdk1.8.0_05 export HADOOP_PREFIX=/opt/hadoop-2.7.0
1356Modify the core-site.xml
1357# cd $HADOOP_PREFIX/etc/hadoop
1358# nano core-site.xml
1359Paste following between <configuration> tags
1360<configuration>
1361<property>
1362<name>fs.defaultFS</name>
1363<value>hdfs://localhost:9000</value>
1364</property>
1365</configuration>
1366Modify the hdfs-site.xml
1367# nano hdfs-site.xml
1368Paste following between <configuration> tags
1369<configuration>
1370<property>
1371<name>dfs.replication</name>
1372<value>1</value>
1373</property>
1374</configuration>
1375YARN configuration - Single Node modify the mapred-site.xml
1376# cp mapred-site.xml.template mapred-site.xml
1377# nano mapred-site.xml
1378Paste following between <configuration> tags
1379<configuration>
1380<property>
1381<name>mapreduce.framework.name</name>
1382<value>yarn</value>
1383</property>
1384</configuration> Modiy yarn-site.xml
1385# nano yarn-site.xml
1386Paste following between <configuration> tags
1387<configuration>
1388<property>
1389<name>yarn.nodemanager.aux-services</name>
1390<value>mapreduce_shuffle</value>
1391</property>
1392</configuration>
1393Formatting the HDFS file-system via the NameNode
1394• The first step to starting up your Hadoop installation is formatting the Hadoop files system which is implemented on top of the local file system of our “cluster†which includes only our local machine. We need to do this the first time you set up a Hadoop cluster.
1395• Do not format a running Hadoop file system as you will lose all the data currently in the cluster (in HDFS)
1396root@ubuntu# cd $HADOOP_PREFIX
1397root@ubuntu# bin/hadoop namenode -format
1398
1399
1400Start NameNode daemon and DataNode daemon: (port 50070)
1401root@ubuntu# sbin/start-dfs.sh
1402To know the running daemons jut type jps or /home/user/Downloads/jdk1.8.0_05/bin/jps
1403
1404Start ResourceManager daemon and NodeManager daemon: (port 8088)
1405root@ubuntu# sbin/start-yarn.sh To stop the running process root@ubuntu# sbin/stop-dfs.sh
1406To know the running daemons jut type jps or /home/user/Downloads/jdk1.8.0_05/bin/jps
1407
1408Start ResourceManager daemon and NodeManager daemon: (port 8088)
1409root@ubuntu# sbin/stop-yarn.sh
1410Make the HDFS directories required to execute MapReduce jobs:
1411$ bin/hdfs dfs -mkdir /user
1412$ bin/hdfs dfs -mkdir /user/mit
1413• Copy the input files into the distributed filesystem:
1414$ bin/hdfs dfs -put <input-path>/* /input
1415• Run some of the examples provided:
1416$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar grep /input
1417/output '(CSE)'
1418• Examine the output files:
1419Copy the output files from the distributed filesystem to the local filesystem and examine them:
1420$ bin/hdfs dfs -get output output
1421$ cat output/* or
1422• View the output files on the distributed filesystem:
1423$ bin/hdfs dfs -cat /output/*
1424
1425Output:
1426Hadoop installation:
1427
1428
1429
1430Create the HDFS directories:
1431Result:
1432Thus the one node Hadoop cluster is installed successfully.
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443EX. NO.: 15 Mount the one node Hadoop cluster using FUSE
1444Date:
1445
1446Aim:
1447To mount the one node Hadoop cluster using FUSE.
1448
1449Procedure:
1450Download the cdh3 repository from the internet.
1451$ wget http://archive.cloudera.com/one-click-install/maverick/cdh3-repository_1.0_all.deb
1452Add the cdh3 repository to default system repository.
1453$ sudo dpkg -i cdh3-repository_1.0_all.deb
1454Update the package information using the following command.
1455$ sudo apt-get update
1456Install the hadoop-fuse.
1457$ sudo apt-get install hadoop-0.20-fuse
1458Once fuse-dfs is installed, go ahead and mount HDFS using FUSE as follows:
1459$ sudo hadoop-fuse-dfs dfs://<name_node_hostname>:<namenode_port> <mount_point>
1460
1461Once HDFS has been mounted at <mount_point>, you can use most of the traditional filesystem operations (e.g., cp, rm, cat, mv, mkdir, rmdir, more, scp). However, random write operations such as rsync, and permission related operations such as chmod, chown are not supported in FUSE-mounted HDFS.
1462
1463Result:
1464Thus the one node Hadoop cluster is mounted using FUSE successfully.
1465
1466
1467
1468
1469
1470
1471
1472
1473`EX. NO.: 16 Write a word count program to demonstrate the use of Map and Reduce tasks
1474Date:
1475
1476AIM:
1477Word count program to demonstrate the use of Map and Reduce tasks
1478
1479PRE-REQUISITE:
1480â— Java version > 1.6 is installed and configured properly
1481â— Hadoop version 2.x is installed with proper configuration and hadoop daemons are running
1482PROCEDURE:
14831. Analyze the input file content
14842. Develop the code
1485a. Writing a map function
1486b. Writing a reduce function c. Writing the Driver class
14873. Compiling the source
14884. Building the JAR file
14895. Starting the DFS
14906. Creating Input path in HDFS and moving the data into Input path
14917. Executing the program
1492Sample Program:
1493import java.io.IOException;
1494import java.util.StringTokenizer;
1495import org.apache.hadoop.conf.Configuration;
1496import org.apache.hadoop.fs.Path;
1497import org.apache.hadoop.io.IntWritable;
1498import org.apache.hadoop.io.Text;
1499import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer;
1500import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.util.GenericOptionsParser;
1501public class WordCount{
1502//Step a
1503public static class TokenizerMapper extends Mapper < Object , Text, Text, IntWritable >
1504{
1505//hadoop supported data types
1506private final static IntWritable one = new IntWritable(1);
1507private Text word = new Text();
1508//map method that performs the tokenizer job and framing the initial key value pairs public void map( Object key, Text value, Context context) throws IOException ,
1509InterruptedException
1510{
1511//taking one line at a time and tokenizing the same
1512StringTokenizer itr = new StringTokenizer (value.toString());
1513//iterating through all the words available in that line and forming the key value pair while (itr.hasMoreTokens()){
1514word.set(itr.nextToken());
1515//sending to the context which inturn passes the same to reducer
1516 context.write(word, one);
1517
1518}
1519}
1520}
1521
1522//Step b
1523public static class IntSumReducer extends Reducer < Text, IntWritable, Text, IntWritable >{
1524private IntWritable result = new IntWritable();
1525// Reduce method accepts the Key Value pairs from mappers, do the aggregation based on keys //
1526and produce the final output
1527public void reduce(Text key, Iterable < IntWritable > values, Context context) throws
1528IOException , InterruptedException{
1529int sum = 0;
1530/*iterates through all the values available with a key and
1531add them together and give the final result as the key and sum of its values*/
1532for (IntWritable val: values){
1533sum += val.get();
1534}
1535result.set(sum); context.write(key, result);
1536}
1537}
1538
1539//Step c
1540public static void main( String [] args) throws Exception{
1541//creating conf instance for Job Configuration
1542Configuration conf = new Configuration();
1543//Parsing the command line arguments
1544String [] otherArgs = new GenericOptionsParser(conf,args).getRemainingArgs();
1545if (otherArgs.length < 2){
1546System .err.println( "Usage: wordcount <in> [<in>...]<out>" ); System .exit(2);
1547}
1548//Create a new Job creating a job object and assigning a job name for identification
1549//purposes
1550Job job = new Job(conf, "word count" );
1551job.setJarByClass(WordCount.class);
1552// Specify various job specific parameters job.setMapperClass(TokenizerMapper.class); job.setCombinerClass(IntSumReducer.class); job.setReducerClass(IntSumReducer.class);
1553//Setting job object with the Data Type of output Key
1554job.setOutputKeyClass(Text.class);
1555//Setting job object with the Data Type of output value
1556job.setOutputValueClass(IntWritable.class);
1557//the hdfs input and output directory to be fetched from the command line
1558for ( int i = 0; i < otherArgs.length 1; ++i){
1559FileInputFormat.addInputPath(job, new Path(otherArgs[i]));
1560}
1561
1562
1563
1564FileOutputFormat.setOutputPath(job, new Path(otherArgs[otherArgs.length 1]));
1565System .exit(job.waitForCompletion( true ) ? 0 : 1);
1566}
1567}
1568STEPS:
15691. Start NameNode daemon and DataNode daemon: (port 50070)
1570$ sbin/start-dfs.sh
15712. Start ResourceManager daemon and NodeManager daemon: (port 8088)
1572$ sbin/start-yarn.sh
15733. Make the HDFS directories required to execute MapReduce jobs:
1574$ bin/hdfs dfs -mkdir /user
15754. Copy the input files into the distributed filesystem:
1576$ bin/hdfs dfs -put <input-path>/* /input
1577Run some of the examples provided:
1578$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce- examples-2.7.0.jar wordcount /Cloud/file1.txt /op1
157916/05/28 14:07:04 INFO client.RMProxy: Connecting to ResourceManager at
1580/0.0.0.0:8032
158116/05/28 14:07:04 INFO input.FileInputFormat: Total input paths to process : 1
158216/05/28 14:07:04 INFO mapreduce.JobSubmitter: number of splits:1
158316/05/28 14:07:05 INFO mapreduce.JobSubmitter: Submitting tokens for job:
1584job_1464422714543_0004
158516/05/28 14:07:05 INFO impl.YarnClientImpl: Submitted application application_1464422714543_0004
158616/05/28 14:07:05 INFO mapreduce.Job: The url to track the job: http://PLLAB-
158749:8088/proxy/application_1464422714543_0004/
158816/05/28 14:07:05 INFO mapreduce.Job: Running job: job_1464422714543_0004
158916/05/28 14:07:10 INFO mapreduce.Job: Job job_1464422714543_0004 running in uber mode : false
159016/05/28 14:07:10 INFO mapreduce.Job: map 0% reduce 0%
159116/05/28 14:07:13 INFO mapreduce.Job: map 100% reduce 0%
159216/05/28 14:07:17 INFO mapreduce.Job: map 100% reduce 100%
159316/05/28 14:07:18 INFO mapreduce.Job: Job job_1464422714543_0004 completed
1594successfully
159516/05/28 14:07:18 INFO mapreduce.Job: Counters: 49
1596File System Counters
1597FILE: Number of bytes read=155
1598FILE: Number of bytes written=229563
1599FILE: Number of read operations=0
1600FILE: Number of large read operations=0
1601FILE: Number of write operations=0
1602HDFS: Number of bytes read=884837
1603HDFS: Number of bytes written=142
1604HDFS: Number of read operations=6
1605
1606
1607HDFS: Number of large read operations=0
1608HDFS: Number of write operations=2
1609Job Counters
1610Launched map tasks=1
1611Launched reduce tasks=1
1612Data-local map tasks=1
1613Total time spent by all maps in occupied slots (ms)=1615
1614Total time spent by all reduces in occupied slots (ms)=1768
1615Total time spent by all map tasks (ms)=1615
1616Total time spent by all reduce tasks (ms)=1768
1617Total vcore-seconds taken by all map tasks=1615
1618Total vcore-seconds taken by all reduce tasks=1768
1619Total megabyte-seconds taken by all map tasks=1653760
1620Total megabyte-seconds taken by all reduce tasks=1810432
1621Map-Reduce Framework
1622Map input records=1
1623Map output records=99348
1624Map output bytes=1282127
1625Map output materialized bytes=155
1626Input split bytes=102
1627Combine input records=99348
1628Combine output records=10
1629Reduce input groups=10
1630Reduce shuffle bytes=155
1631Reduce input records=10
1632Reduce output records=10
1633Spilled Records=20
1634Shuffled Maps =1
1635Failed Shuffles=0
1636Merged Map outputs=1
1637GC time elapsed (ms)=71
1638CPU time spent (ms)=1720
1639Physical memory (bytes) snapshot=454377472
1640Virtual memory (bytes) snapshot=3883495424
1641Total committed heap usage (bytes)=321388544
1642Shuffle Errors
1643BAD_ID=0
1644CONNECTION=0
1645IO_ERROR=0
1646WRONG_LENGTH=0
1647WRONG_MAP=0
1648WRONG_REDUCE=0
1649File Input Format Counters
1650Bytes Read=884735
1651File Output Format Counters
1652Bytes Written=142 user@ubuntu:~/hadoop-2.7.0$
1653
1654In browser type “http://localhost:50070/†Utilities → Browse the file system
1655
1656Result:
1657Thus the Word count program to use Map and reduce tasks is demonstrated successfully.