Attacks on multiple HPC sites

[TLP:WHITE]

This page covers ongoing attacks. As the TLP:GREEN nature of this link was breached and it can only be considered TLP:WHITE now, this page may not be updated anymore, except for TLP:WHITE data (last update: 2020-05-18 11:00:00). For more information regarding ongoing investigations and cooperation possibilities, please contact irtf@mailman.egi directly.

EGI and its supporting organisations deeply care about their international partners and peer infrastructures, and although so far there is no operational impact on EGI, the EGI CSIRT strongly believes in protecting the community and is actively supporting and coordinating the response to the current security incidents affecting the academic and research sector.

This page covers **TWO** security incidents that may or may not be correlated.

Incident #EGI20200421

Summary

A malicious group is currently targeting academic data centers for CPU mining purposes. The attacker is hopping from one victim to another using compromised SSH credentials.

The compromised hosts are turned into different roles, including:

  • XMR mining hosts (running a hidden XMR binary)
  • XMR-proxy hosts ; The attacker uses these hosts from the XMR mining hosts, to connect to other XMR-proxy hosts and eventually to the actual mining server.
  • SOCKS proxy hosts (running a microSOCKS instance on a high port) ; The attacker connects to these hosts via SSH, often from Tor. MicroSOCKS is used from Tor as well.
  • Tunnel hosts (SSH tunneling) ; The attacker connects via SSH (compromised account) and configure NAT PREROUTING (typically to access private IP spaces).

Key points:

  • Connections to the SOCKS proxy hosts are typically done via TOR or compromised hosts.
  • The attackers uses different techniques to hide the malicious activity, including a malicious Linux Kernel Module (https://github.com/m0nad/Diamorphine).
  • It is not fully understood how SSH credentials are stolen, although some (but not all) victims have discovered compromised SSH binaries.
  • At least in one case, the malicious XMR activity is configured (CRON) to operate only during night times to avoid detection.
  • There are victims in China, Europe and North America.

Indicators of compromise

Network based

IPCommentRole in attack
91.196.70.109XMR mining serverCoordinate the XMR activity
149.156.26.227Victim server andromeda.up.krakow.plMalicious IP used for SSH logins + running SOCKS proxy
149.156.26.56Victim server vega.up.krakow.plMalicious IP used for SSH logins + running SOCKS proxy
142.150.255.49Victim desktop UTORONTOSource for attack on .ca hosts
159.226.234.29Victim server at CAS, ChinaMalicious IP used for SSH logins + running SOCKS proxy

TOR hosts (SOCKS proxy users):

51.77.135.89 (also used for malicious SSH logins)
51.15.177.65
51.75.52.118
51.75.144.43
51.79.53.139
51.79.86.181
212.83.166.62

Additional hosts suspected to be involved, as they were found using the SOCKS proxy on a compromised host:

159.226.88.110 (CSNET, China. TCP/44300 access from krakow.pl): Being investigated by CSTCERT
159.226.62.107 (CSNET, China. HTTPS access from krakow.pl): Compromised (XMR mining) and reinstaled by the admin ~2020-05-08 
159.226.170.127 (CSNET, China. TCP/21 access from krakow.pl): Being investigated by CSTCERT
132.230.222.12 (Uni-Freiburg. SSH access from krakow.pl): Investigated (malicious SSH binaries found) (not clear if they are involved in this or just 20200512)
192.154.2.203 (UCLA, USA. SSH access from krakow.pl): Notified.
129.49.37.67 (SUNYSB, USA. Access from a SOCKS proxy): Notified.
129.49.170.118 (SUNYSB, USA. Access from a SOCKS proxy): NOT Notified

Example:

client[4] 51.75.52.118: connected to 159.226.88.110:44300
client[4] 51.75.52.118: connected to 132.230.222.12:22
client[11] 51.75.144.43: connected to 159.226.170.127:21
client[6] 51.75.144.43: connected to 159.226.88.110:44300

It is helpful to also check all existing connections (lsof) and NAT configurations (iptables).

Filesystem

XMR mining, LKM and host configuration

  • Check for Linux Kernel Modules:

Run kill -63 $(random pid) followed by lsmod, then search for modules with names like: diamorphine, scsi, iscsi, readaps

  • Check the content of /etc/cron.hourly/0anacron
  • Check for the following files:
/home/*/.mozilla/xdm
/tmp/.dbs* 
/tmp/.lock
/tmp/aes.tgz
/tmp/db.tgz 
/tmp/dbsyn* 
/tmp/reserved
/tmp/systemdb 
/tmp/updatedb
/tmp/check_power
/tmp/hdshare
/tmp/readps
/usr/bin/on_ac_power (if different from packaged version)
/usr/lib/libocs.so 
/usr/lib64/.lib/l64
/usr/share/aldi.so
/usr/share/sos/rh.pub (if different from packaged version)
/var/tmp/.lock
/var/tmp/.lock/clogs 
/var/tmp/.lock/cpa.h 
/var/tmp/.lock/ologs 
/wlcg/arc-ce1/cache/.cache

Incident #EGI2020512

Summary

A malicious group is currently targeting academic data centers for unknown purpose.

The attacker is hopping from one victim to another using compromised SSH credentials.

Indicators of compromise

Network based

IPCommentRole in attack
149.156.26.227Andromeda.up.krakow.pl (host now cleaned)IP used for SSH logins
202.120.32.231Shanghai Jiaotong UniversityIP used for SSH logins
202.120.58.243Shanghai Jiaotong UniversityIP used for SSH logins
202.120.58.244Shanghai Jiaotong UniversityIP used for SSH logins
2001:da8:8000:6300::/64Shanghai Jiaotong University (?)IP used for SSH logins
159.226.161.107CSTNET, ChinaIP used for SSH logins
159.226.234.29CSTNET, ChinaIP used for SSH logins

It is helpful to also check all existing connections (lsof) and NAT configurations (iptables).

Filesystem

Check for the following files:

/apps/.ior/read/.terma
/apps/.ior/read/.termb
/etc/fonts/.fonts (suid binary for privileged escalation (contains `setgid(0);setuid(0);execl('/bin/bash/', {'bash', NULL}, NULL)`)
/etc/fonts/.low (Log cleaner, capable of removing logs and entries matching specific users (during some time period) in most log files)
/etc/terminfo/.terma
/etc/terminfo/.termb
$HOME/.mozilla/plugins/.fonts
$HOME/.mozilla/plugins/.low
$HOME/.mozilla/plugins/.aa
$HOME/.mozilla/plugins/test
/usr/lib64/.lib/l64
/var/games/.terma
/var/games/.termb

Contact

If you have any additional information or any match at your site, please contact EGI CSIRT at irtf@mailman.egi.eu
In case of TLP:RED data, use GPG: A97F 3BDD F0EE 01A1 176C C13A 93BF 7F91 5696 F750