Archive for the ‘Troubleshoot’ Tag

Remote logins are currently disabled.

Sunday, November 15th, 2015

Our monitoring systems reported errors with a terminal server. I tried remote desktop and received the following error:

Remote logins are currently disabled.

Someone with administrator access disabled the logins with the change logon command.

The management console for reasons not pertinent to this issue was not usable at this time. In order to enable the logins, I needed to a way to submit the command.

Time for our trusty friend “psexec”

psexec.exe “\\<terminal server>\ change logon /enable”

After that we could access the terminal server.

Advertisement

Winlogon initiates shutdown. Reason Code: 0x500ff

Thursday, April 9th, 2015

One machine decided it didn’t want to work anymore. The event viewer showed a message logged where the winlogon initiated a shutdown with the Reason Code: 0x500ff.

Not a typical error.  Power was ok and there wasn’t anything else obvious.

A Technet question found a similar issue happened to someone else.

The power supply was reseated and so far the system remains up.

iLO 3 reports (Error code: ssl_error_bad_mac_alert)

Monday, April 6th, 2015

We are transitioning our console management setup and one of the Windows systems reported an error when trying to access it through the iLO management port.

Secure Connection Failed
An error occurred during a connection to <FQDN>. SSL peer reports incorrect 
Message Authentication Code. (Error code: ssl_error_bad_mac_alert)
The page you are trying to view cannot be shown because the authenticity of 
the received data could not be verified.
Please contact the website owners to inform them of this problem.

It’s an odd error as this was the only system reporting it and it happened on different browsers (IE, Firefox, Chrome).

I tried resetting to factory defaults but that didn’t solve it.

Drivers were up to date but the firmware looked old.  I pulled down the latest version and installed it.

Problem solved.

Correcting a bad superblock on Redhat

Saturday, April 4th, 2015

One system had an issue with the secondary drive. The monitoring system reported it was in a Read-Only state. Suspecting a bad superblock as they happen from time to time; I gave the system a reboot.

As expected; I received:

 *** An error occurred during the file system check.
 *** Dropping you to a shell; the system will reboot.
 *** When you leave the shell.
 Give root password for maintenance
 (or type Control-D)
 (repair file system)#
After entering the root password; it was time to repair. In my case, the problem was easy as the second drive is allocated to one mount. If you are not sure, you will have to look at the partitions.
To list out the partitions, simply enter:
fdisk -l

In my situation, I was interested in this part:

Disk /dev/cciss/c0d1: 146.7 GB, 146778685440 bytes
255 heads, 63 sectors/track, 17844 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

           Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d1p1   *           1       17844   143331898+  83  Linux

If you have multiple partitions, you can verify the label as a precaution (that is if it’s still intact) by using the e2label command. For example:

e2label /dev/cciss/c0d1p1

Having verified the partition, it’s time to restore the superblock by using backup. This is accomplished by using the mke2fs command. Note: if the OS is running, you will need to unmount the drive.

Example:

mke2fs -n /dev/cciss/c0d1p1

It will list output (I forgot to copy it), but at the bottom you will see:

Superblock backups stored on blocks:
(various numbers)

It doesn’t matter which one you choose; I picked the third one. To start the restore of the superblock; enter:

e2fsck -y -b <block number> /dev/your drive path

The “-y” option is a good idea if your partition is large. Otherwise, you will find yourself pressing the “y” key many many times.

Once it’s completed, reboot the system.

Don’t be surprised if it doesn’t solve it on the first pass. Simply re-list the backup superblocks and use a different one for the e2fsck command. In my situation, it took three attempts.

Much as I would like to impress you with my knowledge, I have to give people their acknowledgments for reviewing their blog and site for review.

Linux Expresso

Linux Forums

The Windows Modules Installer Service keeps starting and stopping.

Friday, December 12th, 2014

While checking a performance issue on a Windows 2008 server; I noticed the system log had these messages logged every three minutes or so.

The Windows Modules Installer service entered the running state.
The Windows Error Reporting Service service entered the running state.
The start type of the Windows Modules Installer service was changed from /
   demand start to auto start.
The start type of the Windows Modules Installer service was changed from /
   auto start to demand start.
The Windows Modules Installer service entered the stopped state.
The Windows Error Reporting Service service entered the stopped state.

The application log showed these messages roughly the same period of time (I didn’t seriously compare):

Windows(R) Lightweight Directory Access Protocol (LDAP) failed a request to connect to /
   Active Directory Domain Services(R) for Windows user <domain\user>.
Without the corresponding UNIX identity of the Windows user, the user cannot access /
   Network File System (NFS) shared resources.
Verify that the Windows user is in Active Directory Domain Services and has access permissions.

I spoke to the users and they reported a previous experiment with NFS and they no longer needed the NFS service or client anymore. I removed the role and disabled the two services (server needed a reboot but it was in use).

The messages stopped.

Not sure why the modules installer was getting invoked and it wasn’t worth researching at this point.

I may look into it later.

saveIndx: Unknown index name from ELIM

Monday, August 11th, 2014

While reviewing a server for an issue; I noticed a few messages from LSF.

saveIndx: Unknown index name <name> from ELIM

This message is reported a missing flag from lsf.shared. A quick edit and a process and service restart and the message went away.

 

FMD log files are large!

Friday, August 8th, 2014

A trouble call reported an old server running Solaris 10 with a full root partition.

Looking around I noticed the FMD log files were quite large and the active file was growing.  To ease the space condition, I deleted the rotated files (ie .0, .1, etc.) and started looking into what was the problem.

The Solaris Fault Management Facility was created to provide a self-healing capability.  It through the fmd daemon monitors various aspects of system health and as in this case, logged many messages for system issues.

The first obvious check was to use the fmadm faulty command to see if anything was flagged as faulty. In this situation; there was a bad dimm.

This wasn’t enough to fill log files so I had a look at /var/fm/fmd file and it had several entries for a processor.

The fmstat command which will report statistics logged by fmd and it’s modules confirmed the log activity.

Since it was an old server with no warranty; the hardware people were notified to look at the server and retire it if they couldn’t repair it.

Please wait for the System Event Notfication Service…

Thursday, August 7th, 2014

One of the most annoying things is a server which is hung up and won’t allow you to access it remotely. Especially, if it requires driving to get to the server.

Such was the case of a trouble call from a user trying to access a server. He would try remote desktop and received the following message:

Please wait for the System Event Notification Service…

The login process would remain at this point.

Remote Reboot failed but the system would ping and I could mount a hidden drive.

I needed remote desktop to work so I could look into this problem.

The System Event Notification Service as described from an article in the Microsoft Developer Network states:

Applications designed for use by mobile users require a unique set of connectivity functions and notifications. In the past these individual applications were required to implement these features internally. The System Event Notification Service (SENS) now provides these capabilities in the operating system, creating a uniform connectivity and notification interface for applications. Using SENS developers can determine connection bandwidth and latency information from within their application and optimize the application’s operation based on those conditions.

It sounds like something that is not needed, but it wasn’t the time to make this call when there was an annoyed user waiting for the server. I suspected we could at least kill the process and see if Remote Desktop would work again. Time to use the useful SC command:

sc \\servername queryex SENS

This returned:

SERVICE_NAME: sens
TYPE               : 20  WIN32_SHARE_PROCESS
STATE              : 4  RUNNING
(STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
WIN32_EXIT_CODE    : 0  (0x0)
SERVICE_EXIT_CODE  : 0  (0x0)
CHECKPOINT         : 0x0
WAIT_HINT          : 0x0
PID                : 976
FLAGS              :

Using the PID; I could attempt a taskkill of the sens process.

taskkill /S <servername> /PID 976 /F

This made the login session continue and I had access to the server.

Checking the Net; there are a several situations which could cause this condition.  It ranged from removing Live Messenger (which was not installed), registry changes (which didn’t work or didn’t apply) to removing an update (which was not installed).

I had to log out and found the error was back.  I used the same taskkill steps and was able to access the server again.

Not seeing the obvious; I figured I would try the “fix most things Microsoft” solution and rebooted.

The error went away.

Sometimes a reboot is all that is needed.

 

Missing license server for windows 2012 Remote Desktop Services

Wednesday, August 6th, 2014

Windows Server 2012 can be a little disconcerting due to the new look and layout.

Such was the case for a request of a 2012 server with Remote Desktop Services enabled. I installed the OS and added it to AD. I installed the requested applications and then I went through the wizard and installed the “Remote Desktop Session Host.”

I didn’t need the license server installed as I have a couple already in place.

When it came time to configure verify the setup; I went to Administrative Tools and looked at Remote Desktop Services and only found “RD Licensing Diagnoser”

Ok? Where do I configure the license servers?  Oh wait! the Dashboard.  I looked under that and found the same thing.

I reinstalled the role and found nothing changed.

I did get the warning about licenses and so I ran the diagnoser to see what was wrong.  Two license servers were found but the problem was the missing license server had the licenses for 2012.

Time became an issue and I literally had the user panicking to use the server so I needed a quick resolution as this system was a short term “crash and burn” setup.

I decided to add the missing entry via the registry and reboot.

The license server list can be found here:

HKLM\SOFTWARE\Wow6432Node\Policies\Microsoft\Windows NT\Terminal Services

The entry you need to modify is : LicenseServers

I added the missing license server to the list and rebooted.

The warning message about licenses went away and I verified multiple users could access the server.

Time to crack open the 2012 books and papers!

 

Can’t remote desktop to a windows 2008 R2 server

Tuesday, June 17th, 2014

This was a strange problem. I have a simple Windows 2008 R2 server with remote desktop services installed. Everything was properly configured. Plenty of licenses and diagnostics reported the license server was available.

The problem? Can’t access the server through remote desktop. No errors in the logs and the only message available to me was the generic:

Did all the noob checks and I even reloaded Remote Desktop Services, removed and re-added the server to AD.

Still no access.

I installed tightvnc as I did not want to live at the machine and found I had remote access.

I checked the registry to see if port 3389 was configured and it was.

I used portqueryui to see if the port was in use and it reported:

TCP port 3389 (ms-wbt-server service): NOT LISTENING!

A quick telnet to port 3389 confirmed this.

The firewall was not running and there wasn’t a web server or any other process trying to use that port and thus block Remote Desktop Services. I was tempted to declare too much time debugging this and reload but I decided to dig around the Net a little more and stumbled on an old discussion on technet.

As mentioned in the post by itdoug; I found the hidden device driver called “remote desktop services security filter driver” disabled (You just bring up the Device Manager, select show hidden devices and look under Non-Plug and Play Drivers).

I tried to enable it and it failed. I then uninstalled it and rebooted.

Remote Desktop worked after that.