Tuesday , 16 July 2019
Breaking News

Automation

Yarn Architecture/Job Submission Procedure

YARN Architecture => https://www.ibm.com/developerworks/library/bd-yarn-intro/ Key point – ResourceManager takes care of the ApplicationMasters, while the ApplicationMasters takes care of tasks. Application Master – One per YARN application – Runs in a container – Framework/application specific – Communicates with the ResourceManager scheduler to request containers to run application tasks – Ensures NodeManager(s) complete tasks ResourceManager – One per cluster– Initiates application start-up – Schedules …

Read More »

Automation using PSSH/PSCP/SSHPass

PSSH By default, pssh uses at most 32 ssh processes in parallel to ssh to the multiple servers. (This is important in case you’re controlling hundreds of servers.) By default, it also uses a timeout of one minute to ssh to a node and obtain a result. For ssh commands that take longer than this (e.g., sleep 61), the -t …

Read More »