During the installation of pushing an Oracle agent from Enterprise Manager 13c to a host the initiaal handshake failed, PROV-16011: Algorithm negotiation fail
In the logfile (/u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/agentpush/2016-02-17_13-46-28-PM/applogs/myhost_deploy.log) which is created when an agent push is started I saw the following error reported:
2016-02-17_13-58-34:INFO:============Creating Deployment Vo for key : Initialization and setting status as NOT_EXECUTED
2016-02-17_13-58-34:INFO:=====Creating Action Vo for actionSSHValidations
2016-02-17_13-58-34:INFO:=====Creating Action Vo for actionCopyAgentImage
2016-02-17_13-58-34:INFO:deploymentVo has now status: NOT_EXECUTED
2016-02-17_13-58-34:INFO:==SUDO EXISTS false SUDO PRIV false
2016-02-17_13-58-34:INFO:====CALLING ACTIONdoSSHValidations
2016-02-17_13-58-34:INFO:updating status IN_PROGRESS for action name SSHValidations with key: SSHValidations
2016-02-17_13-58-34:INFO:action:NodesAliveperform:true
2016-02-17_13-58-34:INFO:===VALIDATION===:Checking for Nodes Alive
2016-02-17_13-58-34:INFO:RESULT 0
2016-02-17_13-58-34:INFO:====CALLING ACTIONdoNodesAlive
2016-02-17_13-58-34:INFO:action:RemotePasswordCheckperform:true
2016-02-17_13-58-34:INFO:===VALIDATION===:Checking Password Validity on target nodes
2016-02-17_13-58-34:INFO:NODES=myhost Retry Count 5
2016-02-17_13-58-34:INFO:retry count : 1
2016-02-17_13-58-34:INFO:Running cmd /bin/true on node myhost
2016-02-17_13-58-34:INFO:ssh connect timeout 60000
2016-02-17_13-58-37:INFO:Error Message: PROV-16011: Algorithm negotiation fail
2016-02-17_13-58-37:INFO:Error Level : -1
2016-02-17_13-58-37:INFO:Cause : null
2016-02-17_13-58-37:INFO:Full Message: Algorithm negotiation fail
2016-02-17_13-58-37:INFO:Message: Algorithm negotiation fail
2016-02-17_13-58-37:INFO:AuthFailMsg to check password Auth fail keys Auth cancel
2016-02-17_13-58-37:INFO: SSH Connect Failure for user bbuning_admt on node myhost
2016-02-17_13-58-37:INFO: Jsch Validation Failed Problem :SSH connection check failed Recommendation: Ensure that SSH daemon (sshd) on the target host is able to respond to login requests.
2016-02-17_13-58-37:INFO:Breaking since the jsch validation has failed
2016-02-17_13-58-37:INFO:InvocationTargetException Exception
2016-02-17_13-58-37:INFO:Printing Exception :java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at oracle.sysman.core.agentpush.ui.deployer.BaseDeployerOps.executeActions(BaseDeployerOps.java:2156)
at oracle.sysman.core.agentpush.ui.deployer.InitializationDeployer.deploy(InitializationDeployer.java:400)
at oracle.sysman.core.agentpush.ui.deployfwk.DeploymentWorker.run(DeploymentWorker.java:26)
at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:311)
Caused by: oracle.sysman.core.agentpush.ui.exception.JschValidationException
at oracle.sysman.core.agentpush.ui.deployer.JschValidationOps.doRemotePasswordCheck(JschValidationOps.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at oracle.sysman.core.agentpush.ui.deployer.InitializationDeployer.doSSHValidations(InitializationDeployer.java:135)
... 8 more
The message: INFO:Error Message: PROV-16011: Algorithm negotiation fail the ssh connection was giving issues, but it was initieel unclear what was causing this. Checkout MOS I found several issues/solutions reported in relation to the PROV-16011 message. But know of them where the case in this situation.
But it was clear the issue was related to ssh protocol usages, their was some kind of mismatch.
Their was one reference to a bug, which was closed as not a bug and also no solution/workaround mentioned. But it triggered me to check the sshd_config.
The systems are rolled out using puppet and the ssh_config file looks like this:
cat /etc/ssh/sshd_config
#
# /etc/ssh/sshd_config: Managed by Puppet.
#
Protocol 2
SyslogFacility AUTHPRIV
LogLevel INFO
PermitRootLogin no
MaxAuthTries 4
IgnoreRhosts yes
HostbasedAuthentication no
PermitEmptyPasswords no
PermitUserEnvironment no
Ciphers aes256-ctr
ClientAliveInterval 86400
ClientAliveCountMax 0
PasswordAuthentication yes
PubKeyAuthentication no
RSAAuthentication no
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS
X11Forwarding yes
Subsystem sftp /usr/libexec/openssh/sftp-server
In the sshd_config Ciphers aes256-ctr was mentioned. Which indicates that we use this Ciphers as default. In the past the agent push was not able to deal with all the possible Ciphers, so could this still be the issue with the 13.1.0.0 agent?
I comment-out (temporarly) this entry in the ssh_config on both OMS host as well the new host and restarted the sshd daemon.
(T1) root@myhost:~ # vi /etc/ssh/sshd_config
# Ciphers aes256-ctr
(T1) root@myhost:~ # systemctl stop sshd
(T1) root@myhost:~ # systemctl start sshd
(T1) root@myhost:~ # systemctl status sshd
After the steps where performed I was able to push the agent to the host in question.Please be aware that puppet can undo your modification to the ssh_config file.
Hope this helps others when running into this issue.
Remark: for security reasons the hostnames mentioned where renamed from the original name.