The Last Post

This is the last post by me to this web-site.

As such, I will keep it short. :-)

I have enjoyed my time at Glasgow University. I have met some great folks, had some belly laughs, and managed to keep the systems going for most of you for most of the time.

I am going to call that a win. :-)

I move on from here to an unknown future that I am sure will include contact with many of you again. I look forward to that.

Thank you all for your time that you spent with me over the last six+ years, and I wish you all the best for the future.

Shane.

Around 4:30pm on the 26th of July there was another power cut for the Kelvin Building.

Details are sketchy at the moment, but it appears that power was restored some time later by “re-routing” within the University campus. I don’t know what the implications of that are at the moment, but those are the reports coming from the people on the ground.

Due to the uncertainty surrounding the restoration of power (and the fact that we inevitably lose some kit when the power “bounces” ) it was decided to wait until the facts were better understood before bringing the network and the servers back on-line.

UPDATE:
The power was “re-routed” by Scottish Power last night to bring the Kelvin Building (and others) back up. They did this by “hitching” to an existing, unaffected Bus-Bar to supply us with power. Which means that at some point in the future these changes are going to be reversed. When that will be is unknown, but we should get advance notice...

We are now at the stage where I am choosing people and requesting that Computing Services change their email over.

While I would have liked everybody to have voluntarily moved over before the cut-off date, this did not happen.

So here’s what is happening now.

I am selecting (alphabetically) a number of people from GES and Physics each day and submitting their details for the email change.
The idea is to have all the accounts transferred by the end of this week.

When chosen, your @glasgow addressed emails will NOT be re-directed to your @ges/@physics mailboxes and will, instead, remain in your @glasgow mailbox.
Later that same day, your @ges/@physics email will start arriving in your @glasgow mailbox also.

To access your email from that time on, you should use either the Outlook web access form from here:
https://mail.campus.gla.ac.uk
or your properly set-up email program - some directions for setting up various clients are here:
http://www.gla.ac.uk/services/it/forstaff/usinge-mail/exchange/2007/clients/

If you are at all confused by this, please read the posts on this web-site (the title for relevant posts is “Moving Email”)
and look in your inbox for any emails entitled “Email Changes” as these may give you a more complete picture.

Now that you have been moved to the central email service, what does that actually mean?

1. Your ability to send emails through the @ges/@physics will come to a halt over the weekend of the 23rd/24th July
2. Your old email will become “read-only” - that means that you cannot delete it, move it (which is a copy then a delete), or re-arrange it. You can still read it, and you can copy it over the the new server. The Email Transition Plan sets out a 5 year period for this email to be available to you (nominally). This will happen over the next week or so after you have been “switched”.
3. If you want to get your favourite email program working with the new set-up, check the links above or get in touch with your local IT support.

Of the many people that have changed over voluntarily, 3 have had some problems which were fixed and we have one on-going issue.

Overall, I think this is a better result than other Schools that have tried the “all-at-once” method.

There is some valuable information on the “Email Setup” page (link to the left) detailing some settings for Thunderbird 3 and it may be helpful for some mobile devices as well.

Also, Jamie Scott (IGR) has graciously allowed me to publish his email overview of the whole process - all the necessary information in one place!

Please check it out first as it could answer many of your questions at once.

Laptop users should now be able to access to all services (Web, email, etc.)

Any user whose laptop was “trusted” and now appears not to be, please contact the person who got it “trusted” for you in the first place - Sorry, but the list of “trusted” machines on the backup was a little out of date.

Yet again we have had an unplanned power cut.

At 8:42 (as reported by our monitoring UPS) on Friday the 29th of April, workers working on the power distribution in the Bower Building turned on their latest effort to fix the on-going power problems in our corner of the campus, and promptly shutdown the Kelvin, the Bower, and parts of Professor Square.

About 10 minutes later they restored power - however there were consequences.

- In order to protect the mail systems from the ever more frequent power outages I was half way through putting them into a configuration where they shutdown when the power goes out, and won’t come on again until the power is brought on by design. This meant that some of you were able to access your email, others weren’t. Email was not able to leave our systems, so all those who set up forward addresses would not receive any mail.

- Laptop users will be inconvenienced over this weekend due to the loss of the machine responsible for configuring the “trusted”, “untrusted” and wireless networks. I am working on a replacement as I write this.

- We lost another switch.

The irony of the situation is that there was a planned power cut for this same day for the Kelvin building postponed in mid-week.

Email Store size:

It has been pointed out to me that my previous update on this matter ( the post below ) was inappropriate in the following areas:
1) It named an individual as being a person responsible for a central service, while it is actually a team of people that are responsible for Central Services.
2) It implied that the named individual had agreed to something that they had not actually agreed to.

I unreservedly apologise for any embarrassment caused, it was not my intention to embarrass anyone, only to impart some concrete information into a process that is still taking shape.

What you need to do now

Some of you have contacted me with questions about what you actually need to do.

The short answer is “nothing now”. Generally, your Exchange mailbox does not exist yet, so you cannot connect to it.

I have not yet received the spreadsheets (mentioned in the Transition Plan) back from CS, and I do not expect to have them before early May so I cannot proceed with arranging the second phase of the plan until they arrive. When they do arrive, the inactive “forward” file will be created - you will be notified via an update on this website - and then information regarding how to connect to your Exchange mailbox from various mail clients will be made available.

At some point in the future ( before 30th June ) and after your Exchange mailbox has been created you will need to contact me to activate the forwarding mechanism for your email, so that I can request and co-ordinate the change via the help-desk.
If you do nothing, then after the 30th of June Computing Services will no longer forward your @glasgow email to your @physics account, and your @physics account will forward any incoming email to your @glasgow account. You will you not be able to send email through the physics servers.

So, effectively, you have a window of approximately 6 weeks to sort out the sending and receiving of email, but a lot longer (nominally 5 years) to decide where you are going to store your “legacy” email.

Further updates will be posted as information becomes available.

Please feel free to call if there is anything that you feel needs clarified.

Shane.

Email Store size:

After speaking to Gavin Watt , (the man when it comes to the University Exchange email system), it has been agreed that the initial amount of storage that will be allocated to each person will be: ( current mail store size +10% )

The spreadsheets referred to in the email transition plan Step 1 have been prepared with that extra information and are now on their way to the relevant people in IT Services.

This means that you can now plan how to move your email without space constraints and in a manner that suits you and your schedule.

Post Graduates email:

Some Post Graduate students are unclear about what is happening to their @physics email address - the short answer is nothing, they will continue to receive email - BUT - if you have not given me a valid forwarding address, then your email will be bounced.

A valid forward address is any email address that you can access and receives email - INCLUDING your GUID@student.glasgow.ac.uk or any @gmail, @yahoo, @hotmail or any other email provider address. Please note that if you give me a forward address now, it will be used immediately.

Thanks to all those PG’s who have provided a forwarding address already ( someidentifier.glasgow@gmail.com seems a popular format ) - perhaps you could spur on your colleagues?

Post Graduates NB

If you already have a forward address on your GUID@student.glasgow.ac.uk pointing to your @physics address and you then give me your GUID@student.glasgow.ac.uk> as your forward address for your @physics address “™Bad Things” will happen!

As has been rumoured for some time, email systems run by individual Schools are to be moved to the Central email offering.

Please note: This is not up for debate - it has been decided.

Here is the link to the transition plan - please read it carefully.

The timetable is short (planned to finish by 30th June 2011) , and the processes outlined in the Transition plan are already underway.

How does it affect you?

In the short term: ( month 1)

Processes outlined in the Transition Plan will be underway. They will not change the way you access or send email.

In the medium term: ( month 2 )

Email clients will need to be altered to access the Central Service.

The latest versions of Outlook can access both Exchange accounts and IMAP accounts (our existing type) and will need to be configured to do so. Outlook in “Exchange Mode” is the “richest” client and will give access to all features of the Exchange server.

Other clients (Thunderbird, Mac Mail etc.) can access the Exchange Server as if it was an IMAP server and so can still be used for those who are not on Windows.

The forwarding of legacy email to your @glasgow mailbox is a default - if you wish to have your old email address redirected to somewhere other than your @glasgow address, then you need to notify it-support@physics.gla.ac.uk.

The @glasgow re-direction will happen by default at the end of month 3, as will the change to “read-only” status for your existing email.

Honorary staff without staff numbers will not be given an @glasgow address (to my knowledge) , so will have to supply me with an alternative email address to forward their email - I suggest a “google email” account as it is perhaps one of the better free suppliers.

Postgraduate Students will not have an @glasgow address of the form Firstname.Lastname@glasgow.ac.uk, but will have to use their existing GUID@student.glasgow.ac.uk, unless they wish their legacy email to be re-directed to another email account - in that case they should contact it-support@physics.gla.ac.uk with the new forwarding address.

Other people who have email accounts on the Physics server will need to contact it-support@physics.gla.ac.uk to have their email redirected - any person who does not have a valid forwarding address at the end of June will have their email bounced.

In the long term: ( month 3 and after)

As noted in the Transition Plan, your “legacy” email will remain on the physics server, but in a “read-only” mode - you cannot add to it, you cannot delete it, you cannot send emails through it - but you can transfer it to the Central system (given enough time and available space on the Central servers).

Mailman email lists will remain active, but you should re-join them with your @glasgow address if you haven’t done so already. Mailman list “owners” should think about moving their lists to the central email service as the local email service will not be getting the current level of support.


Some questions staff may have might be answered by this web page:
http://www.gla.ac.uk/services/it/servicedefinitions/staffexchange2007/

Some questions that Postgraduate students may have might be answered by this web page
http://www.gla.ac.uk/services/it/servicedefinitions/studentemail/

This just in from the Powers That Be:
“Some users are experiencing a 'pop up' warning from Sophos telling them that Version 7 is nearing retirement and they should contact their system administrator. There is no need to do this. IT Services is aware of the issue and is working with Sophos and our reseller to resolve it.”

Looks like the whole of Gilmore Hill was affected by 2 power cuts between 3 and 4pm today.

We are still cleaning up after it, but I believe the mail system is now running correctly again (we lost 1 server, and another had disk issues)

Please let me know if you notice a machine or service that is not working as it should.

Lately, a number of wireless access points have been disconnected from the network by (presumably) staff taking out the ethernet cables.

Please don’t do this. The WAPs will continue to allow wireless connections but will not be able to connect to the School network nor the Internet. This gives rise to large amounts of frustration, lots of support calls and very grumpy staff!

If you need a connection for something then please contact the local IT support person for your group, or IT Support in room 238.

From @glasgow ...

As you know, the use of the recently bestowed @glasgow email address is now mandatory for use as both your main receiving address and your main sending address.

I don’t propose to discuss the pros and cons of this here, only to show how it can be done with as little disruption as possible to the normal flow of email in the systems we use.

Firstly, the use of the @glasgow address has been made as simple as possible by the guys at IT Services - simply put, if you use the @glasgow address as your main receiving address, your email will flow (eventually) to your physics mailstore. They have done this by “aliasing” all @glasgow addresses to the existing @physics addresses. There are some downsides to this (see earlier blog posts and emails on the subject), but on the whole, it seems to be working well enough.
Sending with the @glasgow address as your “From” address is very simple if you use Thunderbird, more complicated if you use Outlook and Mac Mail.

I have posted walkthroughs on how to set up Thunderbird, Outlook and Mac Mail. (pdf’s, open in new window).

While I believe the Outlook method will work, I am not sure it is the optimal method. I had thought to use different “profiles”, but this involves re-starting Outlook every time you want to send an email, which I dismissed as too disruptive.

If anybody has any suggestions for Outlook, then please get in touch.

Email Signatures

Lastly, email signatures. Those of you who have not already done so, please change your email signatures to reflect the change in your email address.

“Legacy” email addresses

The use of email systems at the College/School level is the current focus of the College IT Working Party. Discussions are ongoing, and one of the main concerns (yours, as well as mine, judging from the emails I have received on the subject) is the preservation of the @physics @astro et al email addresses that have been active for the last 10+ years. Because this is “simple” in terms of administration and costs and provides many benefits, (like showing a degree of professionalism and continuity, as well as courtesy to readers of older published journals) I am hopeful that our argument will prevail.

What the #!&*! is my email address???

Lastly, a number of people have seem to be rather confused about their @glasgow address. Here is the simple explanation.

Your @glasgow address is of the form firstname.lastname@glasgow.ac.uk . You can check this by looking yourself up on the staff search on the University website.

And please don’t “mix’n’match”!

firstname.lastname@physics.gla.ac.uk is not a recognised email address. (unless you have been told to use it by School IT Support)
firstinitial.lastname@glasgow.ac.uk is not a recognised email address.
loginname@glasgow.ac.uk is not a recognised email address

If you use any of these (and, yes, I have seen them all used) your email will bounce.

Please call if there is anything here you wish to discuss.

Shane Kelly

The Current State of Email

There is a lot happening with email at the present time:
- The new @glasgow addressing
- The new requirement by the centre to send email with the @glasgow return address
- The slow performance of the physics email server
- The consultation on the moving of the email service to the central email service

If you are confused, spare a thought for those of us trying to make it all work! :-)

Taking each of the above in turn, I will try to (accurately and without prejudice) detail what is happening and how it might affect your use of email.

@glasgow

Each person that has a contract of employment with the university has an address of the form Firstname.Lastname@glasgow.ac.uk , except where two (or more) people have the same name - then the person who has been here longer will be entitled to use the name, and the rest will have numbers added to the Firstname.Lastname part. Also, the opportunity to change the Firstname part of the address (for things such as if your formal name is James, but you prefer Jim, etc.) was presented at the start of use of these names, but I can no longer find the option to change this on the HR Self Service website. You may still be able to amend this by contacting HR directly.

Receiving mail sent to this address is automatic - any mail sent to this address is forwarded to your @physics or @astro address and is delivered to your inbox in the normal way.

If you wish to actively promote this address as your current email address you can, in your email program, set the option known as “Reply-To” to your @glasgow.ac.uk address and when people are replying to your email that is the address that will be used.

If you do use this as your “Reply-To” address then it will be subject to the University spam policies first, then ours. Also, it will be impossible to give you any information about any email(s) that you were expecting but have not arrived, or any information about bounced or rejected emails.

@glasgow as your “from” email address

For whatever reason, the centre has decided that this must be done - despite good technical reasons to the contrary. If you are a member of any mailing lists, this new address must be subscribed to the list before you will be allowed to post to the list (most list software works this way, but not all). Since many of you have been members of these mailing lists for a very long time and may not even know how to subscribe with a new email address, this could cause problems. Also, any website where you are registered with your old address may not automatically recognise you with your new address - depends on the website. Any email program rules you may have for dealing with email copied to your own address may need editing and updating.

Computing Service have been asked to report on the take up of this requirement by the end of December.

It may well be that the centre will take steps to enforce this if the take up is not sufficient to whatever criteria they are using for measuring.

There are ways to implement this with minimal impact, and information on how to do it for the majority of email programs will follow in due course.

The slow performance of the Physics mail server.

Our mail server has been overpowered of late and you have been patient with the slow response (especially to saving sent messages) overall. Many of you are now managing your Inbox, Sent/Sent Items, Trash and subsidiary folders much more pro-actively than you have in the past, and attempting to keep them under 1 gigabyte (not always successfully!) which seems to be the maximum size to allow a timely response from the overcrowded server.

Given all that has been happening in the email systems of the University, I was reluctant to spend a great deal of the Schools IT budget on the huge new servers we required which might be made redundant in a short while. Instead, I have implemented smaller ( read cheaper ) servers and they are now in the final stages of testing. By moving selected email accounts onto another server, we should see a return to an acceptable level of email response.

I will shortly be asking for volunteers to move to the new server.

Please note that this does not mean a return to multi-gigabyte folders! Email folders are best kept small for fast response times, easy recovery and backup.

The consultation on the moving of the email service to the central email service.

As many of you know, the drive for efficiency and cost-saving is continuing across the University, and one of the areas being looked at is IT Services. There is a general proposal that all IT services deemed to be “duplication” of a central service be moved to the centre, which will free up “resources” for work on what has been termed “value-add”.

The College of Science and Engineering has made a case for being different, and in that vein, is looking at IT Services where it may make sense to use the centrally provided service rather than the locally provided service and the first of these being looked at is email. All other Colleges are now in the process of, or have completed, the move to the central email service, and some Schools in the College of Science and Engineering are also in the process of moving (carrying on from before the Colleges were formed).

The College is shortly to have a meeting with Computing Services about email to discuss areas of common ground, and identify areas where there may be a need for accommodations.

This process could take some time, and progress will be reported back from time to time.

Please feel free to ask me any questions about anything above.

Shane Kelly

Notice of Shutdown - Sunday 22nd August

For the past 10 days or so, the network has been exhibiting random “freezes” that have resulted in various symptoms - loss of connectivity to “mapped” drives, screens locking up for a short time, some applications randomly closing, etc.

It has taken sometime to arrive at the conclusion that the main backbone switch is perhaps the most likely culprit, and as it is the main switch in the network, it is impossible to comprehensively test without causing a shutdown of all services on the network.

Since the next step in diagnosis of the problem is the replacement of the switch, for which we have to shutdown the network anyway, I have decided to schedule this Sunday from 7am to 9am as the least disruptive time for the replacement process to take place.

I cannot say for sure if this will cure the network problems we have been experiencing, but every test done seems to indicate that the main backbone switch is the one that introduces the faults into the network.

Apologies for the disruption to service.

Shane.

You will have received notification that your “official” email address is now Firstname.Lastname@glasgow.ac.uk - or if you haven’t then you soon will. Read on for the implications of this.....

In the short term

(Now until July 9th)

Do nothing. There is no mechanism in place by which these email addresses can currently work. The target date is the 30th of July for any email addressed to the new email address to actually be delivered.
If you want to alter your assigned email forename, (perhaps from James to Jim) log into the HR self service site and amend your forename there. Please be aware that this may change all your official correspondence to that altered forename. (This is subject to review, but no-one is saying when.)

In the medium term

(July 10th on )

Decide if you want to use this as your main or only email address. The implications can be quite severe if you have memberships on many mailing lists, or have joined on-line forums with your old address.

If you decide NOT to use the new address

Do nothing. Any email addressed to you at your new address will arrive at your existing mailbox, just like any other email. For those of you who have forwarded your email to other accounts, these emails will be forwarded as well.

If you decide you DO want to use the new email address

You will need to change each email client that you use to use the new address as a reply-to address. Once you do this, various email services you use may break. I have no knowledge of your specific email services, so any breakage will have to be dealt with on a case by case basis....

Miscellany

Those people with apostrophes in their name will have the apostrophes automatically stripped out. Those people who have a “van” or “von” etc in front of their surname will have it contracted so it becomes “forename.vanlastname@glasgow.ac.uk”. People with hyphens in their surnames will not be changed.

There are currently no plans to have any role-based email addresses. This should not be a problem, as the old ones will still work.

Call or email me if you have any questions.

The updated certificate is now being used.

You may be asked to accept another certificate (this time for “imap.physics.gla.ac.uk” ), but after it is accepted, you should not be asked again.

Apologies for any inconvenience.

No more global changes are planned for the next two weeks.

After yesterdays changes we found a number of further changes that needed to be done.

The most likely one to have an impact will be done this morning (it is not under our control, so timing may not be exact).

The issues you may see are that you get a “Certificate Warning” stating that the certificate returned is for “pimaps” and you requested one for “imap”.

Please just accept the certificate for now. This problem will be addressed in a future update.

Please call the Faculty IT Office (2956) if you have any questions.

There have been some problems with the switch over.

Usually, these are manifesting as an empty inbox, after you have accepted the new certificate.

These problems are being addressed now, and it may well be that by the time you read this, things are working again.

It does seem to help if you shutdown your mail client and wait a short time ( ~15 minutes ) it speeds up the process, but it has not been necessary in all cases.

Please call if you haven’t got your email back by lunchtime.....

The changes in the email service have been actioned.

There is one symptom (so far) that you may see:

An error message regarding the “Security Certificate” for the server. If you see an error message, please just accept the certificate - it is valid, but the configuration has changed, so it needs to be accepted again.

This may not happen to all email clients.

Please call the Faculty IT office (0141 330 2956 ) if you need further clarification or instructions.

The reading of email has begun to show signs of “hitting the wall” once more.

Unfortunately, just buying a bigger and better server will probably not help as much as I would like, and the benefit is disproportionately low to the expense involved.

What is needed is a different way to manage the huge volume of email that is stored, read and filed each day.

So here is the plan....

A new system whereby we have many smaller servers, each holding a portion of the departments email. The benefit is that new email servers cost less (and can actually be used for longer), any failure to one server will not stop everybody reading their email, and response time to bring up a new server is greatly diminished.

To accomplish this, much must be done on the “back end”, but for you, as an email user, the disruption should be minimal. Unfortunately, for technical reasons, we cannot test this with our normal “load”, but anecdotal evidence and the tests we have done show that the technology works, and works with larger load sizes than we experience.

The first part of the change is to change some “DNS” listings, and since this is something we do not control, the timing of this is a bit vague. I am hoping to get this done early enough in the day so that IT Services support is still around if anything goes wrong and a reversal is needed.
I have scheduled this for Wednesday the 26th of May.

The next part of the change is to remove the dependency on one server for email addresses, aliases and mailing lists. This is a fairly large task that will require some time to complete. Updates will be posted on this web site as and when.

After that comes the actual placement on a new server of several volunteer email accounts, and a period of testing and refining. Hopefully, that will result in a faster email experience for you all.

More information will be posted here as it is needed.

The email servers failing disk has been replaced.

The data is currently rebuilding, but performance is slow.

Spare your blood pressure and put off any emailing until later tomorrow :-)

Hi,
The email server has been having problems over the last few days due to the following single reason:

Users who will not keep their mailboxes to a reasonable size.

One user was attempting to repeatedly open and work with an Inbox of 2.6G and 19000 emails - remotely. Needless to say, this put a huge load on the mail server, and impacted on everybody else trying to use the email server.

That was sorted yesterday, and in a spirit of pre-emptive action, I notified several people with mailboxes above a 1G that they should attempt to reduce the size of these folders.

Strong suggestions were made for this action to be carried out later, but unfortunately many peoples definition of later seems to be “in the next five minutes”.

If you are one of the ones who was notified and hasn’t yet sorted out your over 1G folders, please do not start now. Leave it till later - much later - next week, even.


For future reference, the folders most likely to be problems are INBOX, Sent/Sent Items and the Trash folder (if you don’t empty your “spam” folder regularly also, that could become a problem).

I don’t care how much email you keep, but please use a folder structure, and try to ensure that folders do not grow over 1G in size. That way we can avoid interruptions in email flow like we have had the last few days.

Shane.

Upgrading your Thunderbird 2 installation to Thunderbird 3 has proved to be erratic for several users of Windows and Mac computers.
Symptoms are computer lockup, mail folder hierarchy erratic, folders missing, all email downloaded from the server, and various other inconsistencies.

It is recommended that you do not upgrade at this time.

It would appear that the wireless connection to the web has been restored.

Due to problems with the Central caches, internet connections from wireless access points is intermittent.
Computing Services have been notified

As of last night, new spam detection tools were deployed across the Faculty mail services.

Possible consequences of this are:

1. An increase in the number of emails detected as “probably spam”
Check your spam folder regularly, and perhaps a bit more often in the next few days.

2. An increase in the number of emails rejected as spam
A good thing if they really are spam, but if you are now not receiving emails that you previously got without any problems, please let me know and I will try to track down the problem. As a minimum, I need the following information before even attempting to trace the problem:
- correct sender email address
- date and time of sending (within an hour or so is fine)

3. An increase in spam getting through to you.
Paradoxically, the new tools may result in more spam getting through for a while, until all the adjustments in other spam fighting strategies that we use and the new tools are in sync. Sorry, but that is the nature of what we are fighting. Please report any marked increase in spams that you receive.

In tests, the new tools were more effective, but as we cannot test on live mail streams, test results are an indication, no more, and should be treated cautiously - so please let me know if you think there are problems.

Some figures for your perusal:

Between 19-04-2009 04:41 and 23-04-2009 08:22

connections to "outside" mail gateway:                          74547
delivered by "outside" mail gateway:                            21206
marked as "probably spam" by spam tools:                        1448
rejected connections by the "outside" mail gateway:             66409

Note that a single "connection" may involve multiple deliveries.

At around 2:25 this afternoon, a power supply for one of the main campus routers went down, causing disruption to most services within the campus. The fault has been rectified, and normal service has been restored.

Yahoo (and hence btinternet, which uses yahoo ) freemail service, has for some reason, decided to defer emails from (it appears) all .ac.uk domains.
If you correspond with people using these services, or forward your email to a yahoo account, you may experience, at best, a delay, and at worst total rejection of your email.
From past experience with these types of organisations, contacting them through their support channels is a waste of time - and so it would appear to be this time.
Representations are being made at a higher level, but no response has been forthcoming.
The situation has occurred in the past and has “self-rectified” after a few hours. This does not appear to happening in this case.
More information as it comes to hand.

The network started running slowly last night (17-02-2009) around 11pm.

Diagnostics run by Computing Service this morning pointed to a problem within the Physics network.
Early diagnostics run remotely indicated a switching problem, but that subsequently turned out to be incorrect, and the firewall was found to be the problem.

Some troubleshooting on the firewall allowed for the network to begin functioning at full speed again.

The planned move into Room 243d has been accomplished.
Thanks are due to the Grid guys and Andy Pickford for making room in a timely fashion, and for the co-ordinated effort of the occupants of the Faculty IT office.

I have been involved in three similar moves prior to this one and in each one, the moving of servers that are on 24/7 and under continuou load has led to the loss of varying amounts of hardware and data. This time we did not lose any hardware, but a filesystem refused to mount cleanly and is showing signs of needing restored. This will be taken care of over the next few days.

And we came in ahead of schedule ...

Let’s hope we don’t have to go through it again any time soon!

Last night (10/11/08) at around 4pm, the mail server was overloaded by the repeated sending (unintentionally) of an 8.2Megabyte email (normal emails are around 1kilobyte to 200 kilobytes in size) to several lists and to other departments, totalling over 500 recipients. At last count, there were 16 such emails sent. The resulting load meant that email reading and sending was slowed significantly, until the load was reduced to manageable proportions.
After clearing the email queues of the errant copies, the server was allowed to work through the accumulated backlog, and normal service was restored at around 6:15pm.

Once again the mail server started refusing connections at around 2am this morning (Monday 22/09/2008).

From the investigations I did on Sunday, I knew that the filestore where our email was kept would be partially corrupt, and the only way to fix it was to take the mailserver off-line, and run a deep file system repair tool. ( I ran the quicker version on Sunday (2+ hours) but it obviously didn’t fix things.)
On 300+ gigabytes of mail, experience told me that the time taken to run this would be days.
I therefore took the decision to bring up another server that I had been preparing for another purpose and turn it into our mailserver.
Unfortunately, that meant that the email between our last backup and this morning at 2am would be lost.
It appears that, although some people have been heavily inconvenienced, the majority of you actually took the weekend off, and had very little in the way of email traffic over that period. For those that were inconvenienced, I can only apologise.

It appears that the new server is now settling in and coping with the load.

Background:
The physics email service is made up of an incoming mail server, a mail store and an outgoing mailserver.
The incoming mail server is the one that is responsible for all incoming (to the department) emails. So when you send something from the outside (gmail or yahoo or CERN etc. account) it will first arrive at this server to be virus scanned and spam scored. This server then contacts the mail store, and sends the mail on to it. If the mail-store is not available, the incoming mail server will queue the mail (for up to a week if necessary) until the mail store comes back up.

A connection to the mail store allows you to read and send emails - it is the main component of the email service, and if it has problems, then the effect is to deny everyone mail service until it is fixed.

The outgoing server is where the mail store machine sends its’ outgoing mail for virus scanning, then onto the next server in the chain.

Backups:
The mail store is backed up every second night, but we only keep 4 backups ( giving us a spread of about a week). We simply do not have the space to back up the mail store every night, so the rolling backups and the week spread give us a reasonable margin.

Mail Recovery:
The email that arrived and you stored in your email folders during the period 18th to 22nd September may be recoverable, but due to the length of time it will probably take to recover it, it may not be relevant if recovered. Also it is unknown exactly what may be recoverable.

Once again, apologies for any inconvenience.

The Physics mail server decided to start refusing connections at around 2am this morning.
Preliminary investigations revealed no obvious reason(s). All other servers were operating normally. The email store was checked as far as possible (it is now 300+ Gigabytes which would take many days to completely check!) and all mail (upto around 2am) seemed to be there. It now appears to be operating normally.
Incoming mail was queued on our incoming mail server, and will be delivered over the course of the next few hours, as it catches up.
Investigations will continue into the cause(s) of the problem, and may necessitate a period of downtime (hopefully kept to Sundays) while remedial action is taken.

Faculty IT Manager

There is an ongoing security investigation into an account breach on login.physics this morning.

Check back here for updates during the day (18th Sept)

Final assessment:

No access to the login server until you have changed your password.
To do so, you need to be inside the University. Anyone who is at CERN, DESY, etc will need to phone in to the Faculty IT office to get their password changed. If you are inside the University and don’t know how to change your password, please contact the Faculty IT office.

Latest Update:(Technical)

Information has come to light that indicates this attack is part of an ongoing series of attacks affecting machines on, and talking to the Grid.
This particular attack appears to be a ‘beachhead’ attack, used to probe and attack further into the network. Given the widespread nature of this attack I think we must err on the side of caution and disallow access to the login server for all until you have completed a password change.

Update: (Technical)

Done so far:
- checked all users history files for traces of this attack (no more found)
- checked all binaries looking for replacements/size discrepancies, found 1 symlink installed, traced it back to system update, but checked the binary it was pointing to against a system installed yesterday that is not reachable from outside anyway - md5 sums match
- checked logs on other machines under my control that may have been accessed by robson account

Cautious assessment:

looks like a localized attack that did not elevate privileges (on machines I control - I cannot speak for others).



More details available from Faculty IT for interested parties.

We will be putting in place an updated version of our spam and virus checking engines for the email service later today/over the weekend.
Hopefully, this will not impact on anyone to much. You could see an increase in spam or a decrease in spam, depending on how the new engine(s) work with our particular ‘style’ of email.
The inevitable system tuning will be taking place over the next three weeks or so (more if needed!), so anything that can’t wait should be brought to my attention asap.
Anybody who thinks they have had an incoming email rejected because of the new spam filters should get in touch with the senders details and I will search through the logs to see what has happened.

Some people are reporting identical emails being delivered at almost the same time, 10-15 minutes apart and some are even reporting a third delivery.
Normally, an email appears twice because it was sent twice - i.e. you are on a list which is sent the email, and it is also sent to you personally, and I have found a couple of these types of duplicated emails, but this is certainly not the reason for the majority of double delivered emails we are seeing at the moment.

Preliminary investigation points to a combination of factors including some network timeouts, some impatient sending mail servers and possibly some misconfiguration of the DNS entries.

If you do get double deliveries, and they are not personal emails, email me with all the headers (not the content) of both of them and that will help me track this problem down.
Thanks,
Shane

Hi,
We are being hit by a run of spam with titles like "University Staff now eligible for UK's lowest rate loan" and "Get your Provident Personal Credit Loan today".
Currently, they are not scoring enough to be dropped by the system, but I am working on it.

For Information:
This spam , called 'first run spam' is very difficult to catch the first time it is seen - none of the 'block lists' we use have it registered, it is carefully crafted to appear genuine and it is targeted to a specific industry sector. As the spam run continues, people report it to the block lists, and it is scored higher and just dropped at the gateway. Unfortunately, it does have to be seen at least once for it to be reported. :(

As part of the mail server system checks, a follow up to last weeks checks will be done this Sunday between 10am and noon, meaning that there will be no mail service at that time.
Again, apologies for any inconvenience

The mail server was checked over the weekend and some minor file system problems were encountered.
Accordingly, the mail server will again be taken offline next Sunday (13th April) between 10 and 12 am to rectify any remaining problems.
Apologies for any inconvenience.

This morning, (April 1st - yes, April Fools day!) the mailserver was shutdown and restarted for the first time in 10 months. This meant that the filestore had to be given a cursory check, which took a little longer than it should. Accordingly, the mail server will be taken off-line on Sunday for two hours between 10am and 12am for a more thorough inspection.
Meanwhile, if anybody notes any unusual mail server behaviour , please contact the Faculty IT Office.

Finally!
The disk is in my posession, and can be borrowed from the Faculty IT office from today.

However, you might want to check out this page for installation/upgrade tips and traps before you install ....

The old 'people finder' on the department web site was showing its age.
It couldn't handle the changes to the email accounts without a major re-write, and, as the information is replicated in the Staff Database, it was decided to use that instead.

Now when you click on the 'staff database' button you will be taken to the Staff Database web site where you can enter your query. As a bonus, a printable telephone directory will be available (shortly) so that the once a year issue of the telephone directory no longer needs to happen - you can print your own, or just use it on the web-site.

For those of you who have used the Staff Database, you will know that the details can now be self-managed - i.e. if you change office or telephone number, you can update the database immediately with your new information.

Research group IT people may have to link to the new system if their old system used the 'people finder'. Contact Faculty IT if there are any problems.

The use of Skype on campus is closely controlled due to its rather wide ranging licensing requirements.
The current university policy is here , and if you are going to use Skype, you must tell IT services of your intention.
Laptop users within the department will not know their IP address until it is assigned to them, so I have registered several of the usually assigned addresses with Computing Service.

Welcome back!

The wireless codes for 2008 can be found here. (Department access only)

There seems to be a little confusion about when an email is considered to be "spam".

In our email system, an email is rejected when it receives 8 or more "spam points" as scored by the current rules. If it scores in the range 5 to 8, it is considered "possibly not spam" and it is passed on to the recipient for a final decision. If you have a folder named spam in your email folder list, this email will be placed in that folder automatically, for later review by you. If it scores less than 5 points it is delivered to your inbox. This means that you should review your spam folder regularly because some email that you may consider important may be placed in it. Alternatively, all your mail can be delivered to your inbox by removing your spam folder.

So, when is a delivered email considered spam? When you decide it is, not when the system delivers it to your spam folder.

After a 45 minute meeting with Fiona Duncan of Corporate Communications, I put together a short pdf of the questions I put to her, and her answers.
I must say that the meeting was a positive one, and some willingness to address technical matters was evident.
You can download the file here
Departmental versions of the marque are available from the University Web site, but are difficult to locate as they appear to be at a different address than that advertised. Use the link on the 'Links' page, as that will be updated to reflect the current position.

Today at 8:18 the power to several University buildings (Kelvin, Bower, J.Black, E&B) failed. Immediate steps were taken to reduce the impact of a 'stuttering' start-up and make sure that a power surge (caused by all the computers etc starting up at once ) did not happen.
When this is done, we tend to lose less equipment, but it does mean services can take longer to be restored. Today, PPE systems took longer than the mail and web services due to a start up procedure that needs to be sequenced. Astronomy also had some delays due to a power trip.
The power was restored at 9:34, and key systems were back in place by 9:50.

The recent intermittent network problems were to be due to a failing disk in the firewall(bridge) machine. This machine is a vital part of our defenses against the 'rest of the world' and we cannot run without it. A new unit has been authorised.
Meanwhile, we are working with our substitute firewall(bridge) which was prepared for just such a contingency. It took a wee while to narrow the problem down (we were working - incorrectly , it turns out - on the assumption that the problem was related to the power shutdown over the weekend), but things appear to be returning to normal now.
If you are having mail connection problems, please shutdown your email program and restart it.

For some years now, the Physics and Astronomy Department have been supplying Novell accounts for any GUPHYSICS Windows account, made using an algorithm first laid out by Computing Service many years ago.
It now appears that although these accounts are recognised for some purposes, many (vital) central services now do not recognise these accounts.
Accordingly, we will not be making them up as a matter of course anymore. If there are any staff that haven't got a Novell account made up by Physics, and they need one (it's unlikely), we will certainly make one for you, but it is increasingly clear that Central Accounts will be handled solely by IT Services in the future.

To obtain a Novell account that will allow you to login to Moodle and all other Central Services, follow this procedure:

NB You will need a staff number to complete this form.

- Download this form ( Word PDF )
- Fill it in and FAX it to 0141 330 4850 ( or take it to James Watt North building to the Computing Service service desk)
- Sit back and wait (As this a new procedure, no time estimate is available, but we will update as the time period becomes known)



If you are a (long term) visitor to the Department and you are going to need access to the Central Services, then follow this procedure:

- Download this form ( Word PDF )
- Fill it in and get the Head of Department to sign it
- FAX it to 0141 330 4850 or take/send it to James Watt North building to the Computing Service service desk
- Sit back and wait.

This is the best information we have at this time. If there any updates they will be posted here.

If you are lecturing in the Boyd-Orr building, be aware that some lecture theatres have no automatic access to the internet or web. Your laptop has to be setup with a specific IP address to get any sort of connectivity.
The FAQ will be updated with the information as it comes to hand, so check the FAQ before you go.

If you are travelling abroad and taking a University purchased laptop, you might like to know that the University insurance cover will cover the replacement of the laptop (maximum value £2000), except in the case of theft from an unattended vehicle.
Now, that might seem like a comforting thought, but have you considered the cost and trouble it may take to replace the data from your laptop? Suddenly, that one or two hours spent backing up your laptop before you left doesn't seem like such a waste of time.....
Link to the University Insurance web pages.

The process of getting a new computer account is quite cumbersome.
A new staff member has to traipse around the building, seeing this person and that, collecting log-ins and passwords as they go, usually accompanied by an existing staff member as their guide, who, more often than not is just as confused about the purpose of all the accounts...

So, to help cut down on that tedious process, try this:

Go to the New Accounts page (accessed from the Accounts Navigation button)
Read the notes.
Fill in the form.
Confirm the email
Print the received form.
Get the new staff member to sign it in your presence
Have them bring it to the Faculty IT office (room 238) where they will receive their email address, account(s) names and password.

And, of course, you can start this procedure at any time, even before the new staff member arrives.

Let me know if you think this is easier for all concerned.

An article pointing out some of the common misconceptions surrounding email is available here:
http://www.cio.com/article/128450/ABC_An_Introduction_to_E_mail_Management/1

It makes some very good points, and made me think about just how much the email system does mean to the Physics and Astronomy department.

Spammers are now trying various attachment formats to get through to your mailboxes.
We have so far had .xls (Excel Spreadsheet files), but only in small quantities. These were followed by .zip files containing the .xls spreadsheet, and now we are seeing .gif (a graphic file type) as an attachment. We used to let these through, as they were used in various email signatures.
However, all .gif attachments are now being 'frozen' on the mail server pending review. This may mean longer waiting times for email with these attachments. Apologies for any inconvenience.

The flood of spam hitting our mailserver continues. Favourite type at the moment is a .pdf attachment. While I have implemented some measures to block a lot of these, it appears that some are still getting through (due to 'morphing' or changing the salient features of the original).
If you receive any of this type of spam, just delete it.

The current number of spam emails getting through the filters reflect the major push by spammers world-wide. To quantify this for you, we normally have about 40-50 'active' email connections from the outside world to our email server. Over the past two days that has risen to between 100 and 200 active connections. Many of this spam is 'first-run' spam - i.e spam that has not been reported as such to the blacklists and spam collection sites that we use to help us identify and drop spam - therefore our filters are not able to recognise that this stuff is spam. The good news is that the spam blacklists and collection sites are being fed the information as we speak, and the rate of detection will go up.
Meanwhile, if you get some spam, please attach it to an email to postmaster@physics.gla.ac.uk - please ensure you have all the 'headers' intact, as I can't do anything if they are not. If you don't know what the 'headers' then please don't send the spam, as all the information I need is in the 'headers' .

A new email server has been put in place over the weekend (Sunday, around 12:30). While I don't expect any major issues, a couple of minor ones have been sorted out with help from several people in the department.
As usual, contact the Faculty IT Office if you are having problems.

This is the new ITMatters website, which I hope will become the first stop for anybody looking for information on the IT Infrastructure and services supplied by the Faculty IT Office.

Shane Kelly
Faculty IT Manager