Kompoz Blog   Member Blogs   Groups   Forums    

Jamming
  We Moved!
Sat, Jul 30, 2011 • 01:32 PM UTC - By raf

First, the good news...

We did it.  We moved Kompoz to new bigger and better servers.  You will notice faster uploads, faster browsing (page loads), greatly improved up-time, and more overall stability.  Kompoz is now hosted on Amazon's EC2 infrastructure -- the same infrastructure where amazon.com itself, and countless other high-traffic sites are hosted.   EC2 provides us with scalability, redundancy, and reliability.  We're really excited about this.

Now, the bad news...


We had not planned to move when we did.  We had actually planned to move this weekend, and it was only going to take a couple of hours.  Instead, Murphy's Law kicked our ass.  What happened is so unbelievable.  Four days before we had planned to move, and after four years of running without issue on the same server, the primary disk drive on the server that stored the project art and member profile photos failed.  Just died.  You just can't make this stuff up.

So we were forced to move before we had finished our migration plans.  We had not even finished configuring the new servers at EC2 when the drive failed.  We had to scramble to setup the servers (there are four primary servers), install the operating systems, configure the load balancer, and migrate the database.  And then, during the mad rush, we ran into a snag with the audio transcoder (this is software that allows us to convert WAV, AIF, WMA, OGG, MP3 and other audio files to other formats).  It would not work on the new systems, so we had to totally re-write that component.  

Current State of Affairs

You're reading this, which means that the site is obviously up and running.  The new servers are humming along.  But we're not finished.  There are still things to fix. 

These are the outstanding items:

  • The member photos and album art are glaringly not yet loaded.  As I lamented above, they were on the drive that failed.  We're working with a very good data recovery firm that specializes in recovering data from damaged drives, and we're confident we'll get them back.  Unfortunately, this may take another week or so.  But please, please, please don't wait -- upload new profile photos and album art.  It would really help us out if you could upload new profile photos and album art!
  • Email notifications are turned off for the moment.  This means you will not get an email when new tracks are uploaded to your projects, or when you receive new Private Messages.  We're working on setting up new mail servers.  This should be up and running before Monday.
  • Private Messages sent and new tracks uploaded on Tuesday and Wednesday may be lost.  This is the most regrettable piece of the story.  Our last backup on Monday night was recovered, but the Tuesday backup failed.  So we lost about 24 hours of data.

We're still testing, tweaking, and fixing.  If you notice any problems, please post them on the Tech Support Forum.  I'll be monitoring that today.

Again, I appreciate your patience and I value you as a Kompoz community member!  Please update your profile pics and album art.

Best Regards,
Raf



Share:
URL:


 
Comments

MDK
  MDK on Sat, Jul 30, 2011 @ 02:50 PM UTC

All I can say is, "woo hoo!" Glad to be back.
   
Astronut
  Astronut on Sat, Jul 30, 2011 @ 02:52 PM UTC

Sorry to hear about the problems you ran into. I don't miss IT work whatsoever. That's all part of the "fun," right? :) Good luck with the remaining problems.
   
MattChristensen
  MattChristensen on Sat, Jul 30, 2011 @ 03:01 PM UTC

Man I almost forget how boring life was without Kompoz lol. Thank god it's back. I must've checked every ten minutes since it first went offline.
   
phraser
  phraser on Sat, Jul 30, 2011 @ 03:04 PM UTC

Well done:) Looks like Raf was unable to get rid of the jetson.... (Me)
   
Knight
  Knight on Sat, Jul 30, 2011 @ 03:06 PM UTC

Dude I was like a freaking tweaker with no dope! hehehehe Hope the other problems are minor... Hey Matt, I was doing the same thing... refresh, refresh, refresh.....
   
AliceLeonz
  AliceLeonz on Sat, Jul 30, 2011 @ 03:06 PM UTC

Woooohoooooooo
A million thanks for what you've did...thank youuuuu I can have my play ground back online...
I'm soooo Happyyyyy
   
Brannon
  Brannon on Sat, Jul 30, 2011 @ 03:12 PM UTC

New album art coming right up.
   
Shan
  Shan on Sat, Jul 30, 2011 @ 03:17 PM UTC

Missed you guys! Glad to be back - And better to lose the artwork than the music ;)
   
Lonnie
  Lonnie on Sat, Jul 30, 2011 @ 03:19 PM UTC

wow that was longggg. It's better to have kompoz and no ideas than to have ideas and no kompoz.
   
Pascaloubien
  Pascaloubien on Sat, Jul 30, 2011 @ 03:20 PM UTC

Aaaaa-Uuuuu !!

Are we all so addicted ?? seems to be

Happy works again

   
dazzlefly
  dazzlefly on Sat, Jul 30, 2011 @ 03:21 PM UTC

yippeeee!! Thank you Raf and Tech Dude(s)!

sooo good to be back!

:)
   
PeterWard
  PeterWard on Sat, Jul 30, 2011 @ 03:27 PM UTC

Coolness Raf.

I missed you guys.
   
bjorn
  bjorn on Sat, Jul 30, 2011 @ 03:27 PM UTC

Raf, as we discussed in a recent chat, I have been there, and take this from not only a fellow musician but also a fellow IT "expert": you are a fornicating (dare I say "fucking"?:)) hero! I will not make fun of your mocha-latte frappuchino for at least a week:).

Well done, Sir!

Rock On

B
   
Madmonkey
  Madmonkey on Sat, Jul 30, 2011 @ 03:32 PM UTC

Very happy! I have to say that the site seems to be a lot faster!!
   
minime
  minime on Sat, Jul 30, 2011 @ 03:40 PM UTC

Kompoz is back and this is what counts !

I am Happy !
   
Embers
  Embers on Sat, Jul 30, 2011 @ 04:31 PM UTC

I'm almost over the withdrawal symptoms
   
Fretlessian1
  Fretlessian1 on Sat, Jul 30, 2011 @ 04:34 PM UTC

Good, you're back....now I can give up drinking again!
   
dogbizkits
  dogbizkits on Sat, Jul 30, 2011 @ 04:41 PM UTC

Well done, Raf. Murphy always appears when a critical data migration is ready to go ahead. He always just seems to know when to turn up. Well done again though, Raf. You're the best :o) I also notice a nice speed improvement too.
   
sanger
  sanger on Sat, Jul 30, 2011 @ 05:04 PM UTC

Ahhhhhhh HOME!!!!!!!

Fearless Leader has come thru yet once again!
   
markmiller
  markmiller on Sat, Jul 30, 2011 @ 05:08 PM UTC

Hey...much faster....nice work!
   
DroobyDoobyDoo
  DroobyDoobyDoo on Sat, Jul 30, 2011 @ 05:09 PM UTC

Happy to be back! But why is my hair green?
   
BillOnGuitar
  BillOnGuitar on Sat, Jul 30, 2011 @ 05:13 PM UTC

Ah yes.... Murphy's law. I can relate. Everytime I give level of effort estimate for development work, my boss doubles the time because of this Murphy dude.

The site is much faster. Many thanks for that one!
   
rollingthunder
  rollingthunder on Sat, Jul 30, 2011 @ 06:04 PM UTC

Thanks for the effort getting things back together. I'm glad I'm not doing systems/network mgmt anymore. :)

One thing to note: it's not just tracks that are gone from Tues on, but also the project frameworks themselves. So folks should just eyeball stuff to catch missing projects as well.

Once I'm home, I have a bit of uploading to do.

Again - kudos on dealing with a difficult situation.
   
MulToR
  MulToR on Sat, Jul 30, 2011 @ 06:23 PM UTC

Thanks for your works guys ! Kompoz is back again in France also. I have reupdated my projects picture, not a big deal. Anyway, I could have wait one more week, as now with my 3rd baby (10 days old) nights are too short ;) Cheers
   
JimCavanaugh
  JimCavanaugh on Sat, Jul 30, 2011 @ 06:38 PM UTC

Thanks for all the hard work. So glad to be back!
   
raf
  raf on Sat, Jul 30, 2011 @ 08:03 PM UTC

Thank you all for the support! By the way, if you created a Group here on Kompoz, please upload new artwork for that too.
   
VRun
  VRun on Sat, Jul 30, 2011 @ 09:00 PM UTC

looks like most of our molecules are in tact. ;) Thanks for the great job Raf!

Hey Everybody!
   
Lonnie
  Lonnie on Sat, Jul 30, 2011 @ 09:51 PM UTC

Hey Raf, what happened to the "Donate" button? Can't find it anywhere.
   
raf
  raf on Sat, Jul 30, 2011 @ 09:57 PM UTC

Lonnie, the Kompoz tip jar is on this page. Thanks for the donation! By the way, if you want to donate to Kompoz AND help out a fellow Kompozer at the same time, then consider gifting a "Kompoz PLUS" membership. There's a link on that same page.
   
VRun
  VRun on Sat, Jul 30, 2011 @ 09:59 PM UTC

neato
   
Phoephus
  Phoephus on Sat, Jul 30, 2011 @ 10:48 PM UTC

Hey Raf! Thanks for shutting the site down while I was camping so I wouldn't miss anything.
   
MattChristensen
  MattChristensen on Sun, Jul 31, 2011 @ 01:02 AM UTC

Holy cow the downloads are so fast now!!!!
   
Jeff_Reeks
  Jeff_Reeks on Sun, Jul 31, 2011 @ 01:44 AM UTC

WHEW!!!! .... did a picture fest upload... on all my tracks.....(panting) in record time... Had most of them still.... Good 2 c everyone :)

Does anyone notice yet, comments here.... the the dates and times are off? or ..well what is UTC? I'm so behind :)
   
Lonnie
  Lonnie on Sun, Jul 31, 2011 @ 03:52 AM UTC

UselessTimeCode? It is 8:53 PM Pacific Time

   
SpinDizzy
  SpinDizzy on Sun, Jul 31, 2011 @ 10:59 AM UTC

Wow! You go away for a week to practice with a real life band and come back and the drapes are all changed and your wife looks somehow younger...and they had the good sense to get rid of my ugly mugshot while they were doing the redecorating.

Clowns are well known for their comedic timing!
   
carolb123
  carolb123 on Mon, Aug 1, 2011 @ 01:59 AM UTC

So glad! Kompoz is back! Thank you Raf! :)
   
Dorinda
  Dorinda on Mon, Aug 1, 2011 @ 02:10 PM UTC

Happy are we in this booming community

It is back up again, thank you and amen!

Raf and dudes we appreciate your working hours

So we can all feel that Kompoz power!
   
UkuleleJim
  UkuleleJim on Mon, Aug 1, 2011 @ 03:37 PM UTC

Great work, Raf! Your efforts are greatly appreciated.
   
Sehya
  Sehya on Mon, Aug 1, 2011 @ 06:00 PM UTC

Thanx Raf!!!
Good the site is back :D
   
raf
  raf on Tue, Aug 2, 2011 @ 01:58 AM UTC

UPDATE: Email notifications should be working again.
   
Wire-and-Wood
  Wire-and-Wood on Tue, Aug 2, 2011 @ 02:05 AM UTC

Thanks from me too Raf... It's true, Kompoz is #1 on the web. Tough stuff to live without!
   
rollingthunder
  rollingthunder on Tue, Aug 2, 2011 @ 12:36 PM UTC

I got a notification! Yay! I was going nuts checking all the tunes I've written or worked on for updates...no, I don't have a problem...I can quit anytime...
   
Astronut
  Astronut on Tue, Aug 2, 2011 @ 12:40 PM UTC

Riiiiight. We're ALL innocent in here, buddy.
   
rollingthunder
  rollingthunder on Tue, Aug 2, 2011 @ 05:19 PM UTC

holy donuts of doom....I keep cycling thru tunes looking for updates, forgetting notifications are working... :banghead:
   
UkuleleJim
  UkuleleJim on Tue, Aug 2, 2011 @ 05:32 PM UTC

Notifications aren't working for me, but maybe it's just me...
   
MDK
  MDK on Tue, Aug 2, 2011 @ 05:34 PM UTC

Hmmm... still not working perfectly. A few comments this morning on a private project went unnotificationized.
   
Wire-and-Wood
  Wire-and-Wood on Tue, Aug 2, 2011 @ 05:46 PM UTC

You know what's great for the notifications is the "Dashboard", available from the "my stuff" dropdown. You can configure it to have whatever boxes you want, and mine has one box with "most recent updates", for each projects owned and joined. It's super easy to get an overview of all recent activity
   
MDK
  MDK on Tue, Aug 2, 2011 @ 06:48 PM UTC

That doesn't work for private projects either.
   
rollingthunder
  rollingthunder on Tue, Aug 2, 2011 @ 07:21 PM UTC

Notifications of certain things seem not to work - or maybe it's random and just seems more specific due to the random sampling. Group messages don't seem to be notifying me.

I'll try the Dashboard anyway - been meaning to try and setup something like that.
   
DonnieAlan
  DonnieAlan on Thu, Aug 4, 2011 @ 12:49 PM UTC

I understood a few words of Raf's post, like "the" and "a"...lol
Seriously, Raf and Team...what a job, and I don't envy it, but I AM GRATEFUL for all you've done for the Kompoz Kommunity! Truly grateful.

As it happened, I was away on vacation with my family when all this went down, so I didn't feel a thing!
   

 

Login to Add a Comment


 




     
 
     

 
Recent Blogs by raf
Thank You Pascal
Mon, Mar 4, 2013  •  01:48 AM UTC
 
We're Back Online
Tue, Oct 23, 2012  •  09:40 AM UTC
 
Using Tags to Organize Your Projects
Tue, Sep 18, 2012  •  06:58 PM UTC
 
Introducing 'Kompoz Konnect' Mac App
Wed, Aug 8, 2012  •  02:30 AM UTC
 
Facebook Sharing Update
Mon, Jul 30, 2012  •  12:15 PM UTC
 
Study shows Pop music all sounds the same
Fri, Jul 27, 2012  •  01:38 PM UTC
 
New Hire: Voice Professional
Wed, Jul 18, 2012  •  05:16 PM UTC
 
Kompoz Podcast Needs You
Sat, Jul 14, 2012  •  02:47 PM UTC
 
New Project Licensing Options
Wed, Jul 4, 2012  •  03:14 PM UTC
 
Amazon AWS Outage
Sat, Jun 30, 2012  •  04:28 PM UTC
 
Kompoz Uploader (Windows)
Fri, Jun 29, 2012  •  11:34 PM UTC
 
Kompoz Lab: Sneak Peek
Wed, Jun 27, 2012  •  02:30 AM UTC
 
Fresh Paint
Wed, Jun 20, 2012  •  02:45 PM UTC
 
Happy Father's Day
Sun, Jun 17, 2012  •  04:39 PM UTC
 
Your Kompoz Profile Changed
Sun, Jun 17, 2012  •  03:32 PM UTC
 
Kompoz Hearts Dropbox
Thu, Jun 14, 2012  •  12:37 PM UTC
 
Kompoz HD
Wed, Jun 13, 2012  •  04:15 PM UTC
 
Profile Shortcuts
Tue, Jun 12, 2012  •  12:18 PM UTC
 
Passwords Are So 'Last Week'
Mon, Jun 11, 2012  •  12:10 PM UTC
 
Music: It's in your head, changing your brain
Mon, May 28, 2012  •  02:24 AM UTC
 

More by raf »