DYRL HD Remaster Cleanup Script

[Chad Dunham]

I'm a fan of both Macross and Star Wars, and have just finished a project related to both. I love the clarity of the HD Remaster version of DYRL, but the grain is awful! Over at http://originaltrilogy.com, a forum contributor named G-Force has developed an AVISynth script to cleanup the original unaltered trilogy DVD release from a few years ago, with the aim of eliminating excess film grain and "wobble" from the footage. I took his stabilization script and modified it to work on the MKV file of DYRL. After processing this, I then combined the footage with the Japanese audio, Hikuro's English dub audio, and two subtitle tracks into an "ultimate edition" DYRL. I never realized how bad the FX bootleg of DYRL looked until comparing it to the output of this script.

I don't have any web storage available to share the output, but will be happy to provide it to anyone who can offer it for download. I will post the AVS script here as well, if anyone is interested in using it.

Here are a couple of comparison shots:

[Chad Dunham]

More comparisons. . .

[Chad Dunham]

Here is the script. Copy and paste the text below into Notepad and save with an .AVS extension.

################################################################################

#######

############################ GOUT Filter By G-force V.4.00 ############################

################################################################################

#######

########## Set the path of the MKV file. Use DirectShowSource to work with MKV

DirectShowSource("F:\Videos\Macross Do You Remember Love HD Remastered.mkv")

########## Crop the black borders from the edges of the frames for processing purposes

Crop(24,24,-20,-20,align=true).AddBorders(0,0,0,0)

########## global motion stabilization stage

temp = last.TemporalSoften(7,255,255,25,2)

Interleave(temp.Repair(last.TemporalSoften(1,255,255,25,2)),last)

DePan(last,data=DePanEstimate(last,trust=0,dxmax=1,dymax=0),offset=-1)

SelectEvery(2,0)

########## local motion stabilization/degrain stage

source = last

sigma = 5

filt = source.fft3dfilter(sigma=sigma,sigma2=sigma*.75,sigma3=sigma*.5,sigma4=sigma*.25

,bt=5,sharpen=.3)

Median1(source.RemoveGrain(12),temp,filt)

fw_vec2 = last.MVAnalyse(isb=false,delta=2,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

fw_vec1 = last.MVAnalyse(isb=false,delta=1,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

bw_vec1 = last.MVAnalyse(isb=true, delta=1,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

bw_vec2 = last.MVAnalyse(isb=true, delta=2,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

source.MVDegrain2(bw_vec1,fw_vec1,bw_vec2,fw_vec2,thSAD=400,idx=2)

Median1(source,last,filt,chroma="copy first")

last.MVDegrain2(bw_vec1,fw_vec1,bw_vec2,fw_vec2,thSAD=300,thSCD1=150,thSCD2=75,i

dx=3)

########## sharpening

Contra(last,source)

########## restore stars stage

prest = last

prest.MT_Binarize(threshold=20,upper=true).MT_Expand()

MT_Merge(prest,source,last,chroma="copy first")

########## anti-aliasing

last.EEDI2().Repair(last.NNEDI(dh=true,field=1),9)

########## Resize for DVD resolution

Spline16Resize(718,436).AddBorders(2,22,0,22)

################################################################################

#######

Function Contra(clip denoised, clip orig)

{# re-write of Didée's contra sharpening routine from TemporalDegrain.avs

Interleave(denoised,denoised.RemoveGrain(12,-1),denoised.RemoveGrain(4,-1))

Clense(reduceflicker=false,grey=true)

SelectEvery(3,1)

MT_MakeDiff(last,last.RemoveGrain(12,-1).RemoveGrain(12,-1))

last.Repair(MT_MakeDiff(orig,denoised),1,-1).MT_LUTxy(last,"x 128 - abs y 128 - abs < x y ?")

denoised.MT_AddDiff(last,chroma="copy first")

Return(last)

}

################################################################################

#######

Function Median1(clip input_1, clip input_2, clip input_3, string "chroma")

{# median of 3 clips from Helpers.avs by G-force

chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"

Interleave(input_1,input_2,input_3)

chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)

SelectEvery(3,1)

chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last

Return(last)

}

[Hikuro]

very interesting, Ill have to look at this when I get home for a better look.

So you combined my audio with this new footage and I'm guessing it was "off the hook"?

[Chad Dunham]

very interesting, Ill have to look at this when I get home for a better look.

So you combined my audio with this new footage and I'm guessing it was "off the hook"?

I haven't had time to sit through the whole movie and watch it yes, but so far it's looking (and sounding) very impressive!

[SchizophrenicMC]

very interesting, Ill have to look at this when I get home for a better look.

So you combined my audio with this new footage and I'm guessing it was "off the hook"?

Actually, I think he made a version with the Japanese and HRE, and it was "off the hook".

[boinger]

S-47 Berkut has been getting very good results with Neatvideo sofware for grain removal for his japanese subtitle removal DYRL project.

I'm not sure which software is better without screenshot comparisons.

I keep thinking that the perfect edition dyrl would have been better as the basis for the remastered edition rather than the theatrical edition. But, your version does look like an improvement.

This virtualdub plugin will also come in handy for the remastered SDF tv series that Gubaba and I are putting together.

Thanks!

Edited February 9, 2009 by boinger

[Chad Dunham]

S-47 Berkut has been getting very good results with Neatvideo sofware for grain removal for his japanese subtitle removal DYRL project.

I'm not sure which software is better without screenshot comparisons.

I keep thinking that the perfect edition dyrl would have been better as the basis for the remastered edition rather than the theatrical edition. But, your version does look like an improvement.

This virtualdub plugin will also come in handy for the remastered SDF tv series that Gubaba and I are putting together.

Thanks!

I've heard good things about Neatvideo, but haven't used it much. Here's a version for use with DVD rips of the TV series. It's largely the same, but uses a DGIndex project file as the input and has different crop and resize settings than DYRL. I'm playing around with the Animeigo footage to see if it can be redeemed at all, but there's no way the colors will ever match those of the memorial box set.

And it's actually an AVISynth script, not VirtualDub

################################################################################

#######

############################ GOUT Filter By G-force V.4.00 ############################

################################################################################

#######

Mpeg2Source("G:\Super Dimension Fortress Macross\Animeigo Video\Episode 01 - 0xE0 - Video - MPEG-2 - 720x480 (NTSC) - 4~3.D2V")

Crop(4,0,-2,-0,align=true).AddBorders(0,0,0,0)

########## global motion stabilization stage

temp = last.TemporalSoften(7,255,255,25,2)

Interleave(temp.Repair(last.TemporalSoften(1,255,255,25,2)),last)

DePan(last,data=DePanEstimate(last,trust=0,dxmax=1,dymax=0),offset=-1)

SelectEvery(2,0)

########## local motion stabilization/degrain stage

source = last

sigma = 5

filt = source.fft3dfilter(sigma=sigma,sigma2=sigma*.75,sigma3=sigma*.5,sigma4=sigma*.25

,bt=5,sharpen=.3)

Median1(source.RemoveGrain(12),temp,filt)

fw_vec2 = last.MVAnalyse(isb=false,delta=2,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

fw_vec1 = last.MVAnalyse(isb=false,delta=1,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

bw_vec1 = last.MVAnalyse(isb=true, delta=1,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

bw_vec2 = last.MVAnalyse(isb=true, delta=2,pel=2,sharp=1,overlap=4,plevel=0,idx=1,dct=1)

source.MVDegrain2(bw_vec1,fw_vec1,bw_vec2,fw_vec2,thSAD=400,idx=2)

Median1(source,last,filt,chroma="copy first")

last.MVDegrain2(bw_vec1,fw_vec1,bw_vec2,fw_vec2,thSAD=300,thSCD1=150,thSCD2=75,i

dx=3)

########## sharpening

Contra(last,source)

########## restore stars stage

prest = last

prest.MT_Binarize(threshold=20,upper=true).MT_Expand()

MT_Merge(prest,source,last,chroma="copy first")

########## anti-aliasing

last.EEDI2().Repair(last.NNEDI(dh=true,field=1),9)

Spline16Resize(640,480)

################################################################################

#######

Function Contra(clip denoised, clip orig)

{# re-write of Didée's contra sharpening routine from TemporalDegrain.avs

Interleave(denoised,denoised.RemoveGrain(12,-1),denoised.RemoveGrain(4,-1))

Clense(reduceflicker=false,grey=true)

SelectEvery(3,1)

MT_MakeDiff(last,last.RemoveGrain(12,-1).RemoveGrain(12,-1))

last.Repair(MT_MakeDiff(orig,denoised),1,-1).MT_LUTxy(last,"x 128 - abs y 128 - abs < x y ?")

denoised.MT_AddDiff(last,chroma="copy first")

Return(last)

}

################################################################################

#######

Function Median1(clip input_1, clip input_2, clip input_3, string "chroma")

{# median of 3 clips from Helpers.avs by G-force

chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"

Interleave(input_1,input_2,input_3)

chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)

SelectEvery(3,1)

chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last

Return(last)

}

[boinger]

Thank you again!

[Chad Dunham]

I've done a test encode of about 2500 frames of the first episode of the Animeigo set of the TV show with the script, and am experimenting with some color correction processes on it. The effects of the stabilizing and degraining script is much more subdued on this footage, but it still makes some difference. The color correction is still very much a work in progress. Currently, it just takes an automated white balance filter, then merges the output of that with the original input, so you get a 50/50 blend. To me, the MKV file of the HD remaster memorial set has much better color, but the detail on the download I have seems pretty blurred out. The first frame is the Animeigo ripped video. The second frame is processed through the scripts. The final frame is from the HD MKV file.

Edited February 11, 2009 by Chad Dunham

[Chad Dunham]

Same order as before. . .

[Chad Dunham]

A few more . . .

[promethuem5]

This looks pretty cool, but I am not technologically inclined at all... do you think a new improved edition could be uploaded somewhere using this method?

[Chad Dunham]

This looks pretty cool, but I am not technologically inclined at all... do you think a new improved edition could be uploaded somewhere using this method?

Thanks! An upload project is in the works

[promethuem5]

Glad to hear!

[Mercurial Morpheus]

I've done a test encode of about 2500 frames of the first episode of the Animeigo set of the TV show with the script, and am experimenting with some color correction processes on it. The effects of the stabilizing and degraining script is much more subdued on this footage, but it still makes some difference. The color correction is still very much a work in progress. Currently, it just takes an automated white balance filter, then merges the output of that with the original input, so you get a 50/50 blend. To me, the MKV file of the HD remaster memorial set has much better color, but the detail on the download I have seems pretty blurred out. The first frame is the Animeigo ripped video. The second frame is processed through the scripts. The final frame is from the HD MKV file.

The last one there is a blurred out mess. That would only be good for something like the histograms over at OT.com

Edited February 18, 2009 by Mercurial Morpheus

[Chad Dunham]

Boinger and Gubaba are enhancing the subtitles and muxing together the video I'm running through the script, their updated subtitles and the Japanese audio for the SDFM TV series. You can download the results from here:

http://rapidshare.com/users/SDLIT9

More to come.

Enjoy!

[S-47 Berkut]

My IMHO after closer look on DYRL source

Most hated thing is frame shaking, spend a lots time trying align patches to close burned subs, very boring and then it looks good in static, on moving picture it look like crap.

Problem in cell animation technology, frames are not shake as whole, each cell layer live his own live, +- pixel or half, or quarter. Some times i see local warp effects on frame, may be there cells was not perfectly flat at shot time.

I'm not sure it can be fixed.

About SDF from Animeigo. I think trying clean it is a waste of time. Latest japanese release are superior in detail and color, if colors can be barely matched, but lost details can't appear from script. I think efforts must be focused on getting this release: buy, or find someone with it and trade or persuade. I don't believe that no one on this forum not have it - it will be damn joke.

[boinger]

Re-remastered DYRL

by Chad Dunham

.m2v, 34 parts, HJSplit

The film grain reduction does look better to me.

Thanks, Chad!

There's a dvd subtitle patch in the works, once I get through fixing up an .ass subtitle file.

Then all you would need is the .m2v + patch + retail Bandai Macross DYRL DVD!

************

Thank you, S-47 Berkut for working on fixing up the remastered DYRL!

Just from the test clips, I can see that you put a lot of love into your video upgrading.

You can have a look at what Chad has come up with avisynth on DYRL.

You can use Raptor, which is a good free rapidshare downloading software.

Weblink is in my signature.

S-47 Berkut's dyrl project could be released in the same way as an .m2v + DVD subtitle patch, just to encourage people to buy the original DVD while it's still being sold in retail stores.

Maybe, somebody would like to make custom DVD covers for these DYRL fan edits?

Chad's fixed up Bandai remastered SDF tv episodes are looking better than the original Bandai episodes to me and definitely better than the Animeigo sdf tv episodes. I'm going to be posting up both original and Chad updated SDF episodes in the Macross fan edits thread, so everybody won't feel missed out.

Edited June 22, 2009 by boinger

[nhyone]

Note: I replied here because it is related to DYRL video processing, which is what this thread is all about.

Using Sir Blew's Handbrake settings, I was able to improve video quality and
get the video de-interlaced at the same time on a Techno Boyager R2 DVD.

But, with Sir Blew's settings, the Bandai DYRL R2 DVD still had a significant
interlacing problem where the interlaced frames are ghosted one upon the other
rather than separated into individual frames.

Based on your description, it looks like you are using the 1999 Perfect Edition DVD? That is a nightmare. However, the PS3 can deinterlace it, so it should be doable -- somehow. For HandBrake, you should play around with the detelecine and decomb parameters:

Telecine

Deinterlacing Guide

Decomb

I have not used this DVD as a source, but perhaps I should give it a try one day. I have not taken my DVD out of storage for 10 years. I hope it is still working!

Alternatively, on the original trilogy forum, Dr. Dre (forum name) has been working to get the best
video quality out of DVD using Avisynth and plug-ins. The avisynth software with his
plug in settings take days to encode a few minutes and weeks to encode a whole movie
The changes in video quality from one version to the next are just too subtle for me to
see properly.

That is a very ambitious project! Macross-wise, it is akin to post-processing and upscaling the AnimEigo DVDs to the blu-ray standard...

It is well-worth to get acquianted with Avisynth. HandBrake has limited filters. For maximum flexibility, we need to fall back to Avisynth. I've used it to do simple color-correction. As we know, almost every transfer has its own color! (But we seldom see people arguing over it like grain...)

In the near future, we will be upscaling our 1080p anime to 4K and filters are the key to get good quality.