Author Archives: Tulio Llosa

Kudos to Josh

Josh,

Thank you for all of your assistance on today. After I called for instructions, you went above and beyond and made a personal visit to my office to download and place the Xythos drive on my computer. I truly appreciated you making the effort and taking the time. Thank you again!

——————————————
Dorine J. Brand, Ph.D., M.P.H.
Assistant Professor, Public Health Department
University of Illinois at Springfield
Public Affairs Center 352
One University Plaza, M/S 309
Springfield, IL 62703
Phone: (217) 206-7874
dbrand5@uis.edu

[System Outage] CITES ICCN Networking Layer 3 Equipment Upgrade – Sunday, August 11 from 5:30 AM – 9:30 AM

Begin forwarded message:

From: "Mesplay, Mark" <mmesplay@uillinois.edu>
Date: August 7, 2013, 4:20:18 PM CDT
To: UI-Integrate Event Notice <uieventnotice@uillinois.edu>, "SiteMinder Event Notice" <sitemindereventnotic@uillinois.edu>
Cc: "Smith, Tracy L" <tracys@illinois.edu>, "Skaar, Chris" <cskaar@illinois.edu>
Subject: [System Outage]  CITES ICCN Networking Layer 3 Equipment Upgrade – Sunday, August 11 from 5:30 AM – 9:30 AM

EXECUTIVE SUMMARY:

 

This notice is to inform you of the following scheduled event this Saturday.

 

On Sunday, August 11, 2013 from 5:30 AM to 9:30 AM, CITES will be performing ICCN Networking Layer 3 Equipment Upgrade.  

Per CITES, no impact to Redundant Layer 3 services on the ICCN is expected, however, due to the nature of the work, unintended outages may occur. Non-redundant services should expect several outages during this maintenance window. Optical services on the ICCN will not be affected during the upgrade.  

AITS will be halting Application Manager/Appworx batch processing during this event.

 

 If any unexpected issues are encountered, the System Status page will be updated. 

 

For more detailed analysis of these events, see the appropriate sections below.

 

OF INTEREST TO:

 

Users of the following services:

 

SiteMinder Web Agents may experience brief intermittent periods when they cannot connect to the policy servers.  Websites using CA SiteMinder may experience sporadic  500 errors during these brief intermittent periods.

 

All production Application Manager/Appworx batch jobs scheduled to run Sunday, August 11 between 4:30 AM and approx.10:00 AM.

 

 WHAT & WHEN:

 

Change Date: Sunday, August 11th, 2013 5:30 AM to 9:30 AM – planned CITES ICCN hardware upgrade.

 

WHY:

 

 Per CITES: The reason for this event is to allow engineers to upgrade the Layer 3 hardware on the ICCN.  These upgrades will provide more flexibility to the ICCN as well as provide a 100Gb upgrade path.

 

 

IMPACT:

 

Per CITES:  Redundant Layer 3 services on the ICCN will see little to no impact as a result of the changes.  Non-redundant services should expect several outages during this maintenance window.  Optical services on the ICCN will not be affected during the upgrade.

 

SiteMinder Web Agents may experience brief intermittent periods when they cannot connect to the policy servers.  Websites using CA SiteMinder may experience sporadic  500 errors during these brief intermittent periods.

 

Application Manager processing will be placed on hold from approx. 4:30 AM until 10:00 AM.

 

 

CONTACTS:

 

If you have questions about the CITES event or experience issues as a result, please contact ICCN Operations.

 

Emergency: 217-265-ICCN (4226)

 

If you have questions about halting Application Manager processing, please contact AITS Deployment.

Mark Mesplay

AITS – Deployment

Office: 217-244-2814

Cell: 217-979-7480

Or 

AITS Service Desk:

 (312) 996-4806 (from Chicago)

(217) 333-3102 from Urbana-Champaign or Springfield

 

 

 

[UOFI-BOX] report on yesterday’s Shibboleth outage

Begin forwarded message:

From: "Nye, Mark Allen" <marknye@ILLINOIS.EDU>
Date: July 11, 2013, 1:35:49 PM CDT
To: <UOFI-BOX@LISTSERV.ILLINOIS.EDU>
Subject: [UOFI-BOX] report on yesterday’s Shibboleth outage
Reply-To: U of I Box.net Staff <UOFI-BOX@LISTSERV.ILLINOIS.EDU>, "Nye, Mark Allen" <marknye@ILLINOIS.EDU>

Box’rs,

Yesterday’s UIUC Shibboleth outage was caused by a problem with the terracotta cluster engine, which is used by Shibboleth to share live session data across our Shibboleth IDP nodes. From Keith Wessel:

The standby node lost contact with the active node a couple times during the DC network work yesterday morning. The last time it did, it was half way through a re-initialization of its database. This left it in a hung and uninitialized state. We’ve only seen this once before.



By early afternoon unstable terracotta has consumed enough memory to disrupt Tomcat service on the lead Shibboleth node. The problem was reported to the ID Management group at 2:30pm, and the service was restored ~2:50pm. (Unfortunately response from service management was somewhat delayed due to a fire drill going on in DCL.)

In the short term, Keith will setup a monitor script to watch the terracotta logs and alert on this problem, so next time it can be fixed before it causes a service disruption. In the longer term, the plan is to reconfigure Shibboleth such that terracotta is not required.

best,
Mark

—–
Mark Nye (marknye@illinois.edu)
Collaborative Services Team, CITES AS
University of Illinois at Urbana-Champaign

Campus Internet Connectivity Issue

Internet connectivity is back up at normal speed. Thanks Neil, Chris Mueller and Kara for reporting the problem at around 9:40pm tonight. Thank you, Jamie Voyles, for contacting Kevin and Kevin for going to campus to check the firewall logs.

The internet came back at 11:15pm, but it was still slow and unusable at times. Kevin could see high activity in the firewall logs. By the time Kevin reached campus at around midnight everything seemed back to normal.

We will continue to monitor the issue.

From: Mueller, Christopher Lee
Sent: Friday, March 01, 2013 9:42 PM
To: UIS ITS Staff
Subject: Internet is down

Hey everyone. Neil just called me and internet appears to be down campus wide. If some one could look at this or would be greatly appreciated.

Thanks!
Chris

Portal down Sunday for system maintenance

From: “Nelson, Todd” >
Date: February 10, 2013, 10:53:12 AM CST
To: “Llosa, Tulio C (UIS)” >, “Bellot, Clayton H (UIS)” >
Subject: Re: Portal

Hi Tulio, Yes, there is scheduled maintenance of the portal servers today. There’s more information on the AITS status page at: http://status.uillinois.edu/live/Site.xml?document=SystemStatus.xml. The outage is posted until 6pm, but we should actually be back up around 2pm.

Helpdesk Manager Update

Helpdesk Manager Todd Rotroff is currently on FMLA leave. The responsibilities of the Helpdesk Manager have been assigned to Bill Cox, ITTA, until a permanent Helpdesk Manager can be hired.

Bill has several years of supervisory experience and has a working knowledge of the Desktop Refresh Program. Additionally, Bill has expertise in both computer set up and imaging, as well as demonstrated customer service skills.

We appreciate Bill’s willingness to assume additional responsibilities during this time of transition for the Helpdesk.

Welcome Shellie

Shellie Treat will join the ITS Systems and Services and Vsphere and Security teams as the Assistant Director of Network Services on Monday January 28. She will be responsible for providing leadership and supervision to the teams.

Shellie brings supervisory skills, project management experience, and network security experience to the position. In her roles with the Springfield Clinic, Netconn Solutions and Hanson Information Systems, Ms. Treat demonstrated the ability to lead departments, manage technical projects, create and implement security structures, and provide customer service. She is excited to meet you all and to contribute to our mission.

Please stop by HSB 121 on Monday January 28 to welcome Shellie to our campus.